Analysing and forecasting the first dataset

I will try to write my thoughts white I perform my data analysis.

Importing libraries

In [110]:
import pandas as pd
import numpy as np
from pandas.plotting import lag_plot, autocorrelation_plot
import datetime
from datetime import timedelta
from matplotlib import pyplot
from statsmodels.tsa.seasonal import seasonal_decompose
from statsmodels.tsa.stattools import adfuller
from statsmodels.graphics.tsaplots import plot_acf
from statsmodels.tsa.arima_model import ARIMA
from plotly.offline import init_notebook_mode, iplot
from plotly import graph_objs as go
from statsmodels.tsa.ar_model import AR
from sklearn.metrics import mean_squared_error, r2_score
from sklearn import preprocessing 
from sklearn.preprocessing import OneHotEncoder
from sklearn.cross_validation import train_test_split
from sklearn.feature_selection import SelectKBest, f_regression
from sklearn.linear_model import LinearRegression
from math import sqrt
import bisect
from pickle import dump, load
import warnings
warnings.filterwarnings('ignore')

Making some logics available via functions

In [111]:
def plotly_actual_vs_predicted(index = None, 
                 actuals = None,
                 predictions = None,
                 title = '', 
                 mode = 'lines+markers', 
                 annotations = None):
    common_kw = dict(x=index, mode=mode)
    xaxis = dict(title=title)
    data_actuals = go.Scatter(y=actuals, name='actuals',  **common_kw)
    data_predictions = go.Scatter(y=predictions, name='predictions',  **common_kw)
    data = [data_actuals, data_predictions]
    layout = dict(title=title, showlegend=True, annotations=annotations, xaxis=xaxis)
    fig = dict(data=data, layout=layout)
    iplot(fig, show_link=False)
    
def plot_autocorrelation(values):
    pyplot.figure(figsize=(50, 20), edgecolor='b')
    pyplot.grid(True)
    plot_acf(values.astype(float), ax=pyplot.gca())
    pyplot.show()
    pyplot.clf()

Importing data

In [112]:
dataset1 = pd.read_csv('data1.csv', header=None, names=['valor'])

Quick data check

In [113]:
# Check missing data
print(dataset1.isnull().sum())

# Data distribution
dataset1.describe()
valor    0
dtype: int64
Out[113]:
valor
count 5001.000000
mean 0.000168
std 0.707207
min -0.999986
25% -0.688408
50% 0.005310
75% 0.725324
max 0.999986

Building a time series

Considering that each line is a DAY, we are going to create a logic to define a sequencial index of days (starting from today and going back in the calendar). We dont have necessarily to create these dates, but I think it will help to visualize the serie.

Note: in production datasets it is very common to find gaps in the dataset. If that's the case, I would have to work on inputation to fix these gaps. Here, I dont have this issue.

In [114]:
# Add date to the dataframe
dataset1['col_dt'] = datetime.datetime.now()
dataset1['linha'] = dataset1.index.astype(int)
dataset1['col_dt'] = dataset1.apply(lambda x: x.col_dt - timedelta(days = x.linha), axis=1)

# Store my TS on "ts1" object
ts1 = pd.Series(index=dataset1.col_dt, data=dataset1.valor.values)

# Check some records
print(ts1.head())

# Sanity check (make sure we did not mess the data)
data = {'col_dt': ts1.index, 'valor': ts1.values}
ts1_df = pd.DataFrame(data=data)
sanity_df = pd.merge(dataset1, ts1_df, on=['col_dt'], how='inner')
print('Sanity test: {} inconsistencies'.format(len(sanity_df[sanity_df['valor_x'] != sanity_df['valor_y']])))
col_dt
2019-11-10 04:47:25.022687    0.841471
2019-11-09 04:47:25.022687    0.873736
2019-11-08 04:47:25.022687    0.902554
2019-11-07 04:47:25.022687    0.927809
2019-11-06 04:47:25.022687    0.949402
dtype: float64
Sanity test: 0 inconsistencies

Starting decomposition

Lets analyse the following components:

  • Observed: the average values of the serie
  • Trend: increasing, decreasing pattern (sometimes there is no trend)
  • Seasonality: REGULAR peaks on specific period of times (sometimes there is no seasonality)
  • Noise: which we cannot explain

Sometimes we find isolated peaks which cannot be captured in the model. On such cases, we might want to threat those peaks as "outliers".

It's also worth to highlight that we can use additive or multiplicative approaches to decompose the serie. Additive models suggest that your time series ADDs each component to explain the target variable:

y(t) = Level + Trend + Seasonality + Noise

while Multiplicative models suggest that your time series MULTIPLIES each component to explain f(x):

y(t) = Level Trend Seasonality * Noise

In this case, I am using additive approach because, looking to the data, I dont see huge changes on each value of Y, which makes me think this is closer to linear functions rather than quadract or exponential functions.

In [149]:
# Perform naive decomposition (check package documentation for more info https://www.statsmodels.org/stable/generated/statsmodels.tsa.seasonal.seasonal_decompose.html
# We can refer to each result by .trend, .seasonal, .resid and .observed
result = seasonal_decompose(ts1, model='additive')
result.plot()
pyplot.show()

Looking to these components, we cannot see any trend or seasonality in the serie. The pattern is always constant. Such behavior makes me think this is an stationary serie, since the average value of the serie is remains constant. Lets check.

Stationary test

In [116]:
stationary_test = adfuller(ts1, autolag='AIC')
stationary_test
Out[116]:
(-80.59524128452975,
 0.0,
 5,
 4995,
 {'1%': -3.43165984259144,
  '5%': -2.8621188086591505,
  '10%': -2.5670781005730454},
 -204412.37483308825)

For each of those confidence intervals (99%, 95% and 90%), IF the critical value is greater than your test statistic (-80.59), then the serie is stationary. In this case, this serie is stationary with 99% of confidence

Finally, you we might want to check autocorrelation to guarantee this is not a wite noise (which does not make sense, based on the evidences so far)

Autocorrelation

Checking ACF plot

In [117]:
plot_autocorrelation(ts1.values)
<Figure size 432x288 with 0 Axes>

Because we see many points out of the blue zone, we can assume this is autocorrelated. If you are not sure, we can take a look on other visualizations. For example, pandas has a built-in function where we can plot t and t+1 lag variable with minimal effort in terms of data transformation.

In [118]:
lag_plot(ts1)
Out[118]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f40dd976748>

This is definetelly high correlated. One last test is actually getting the Pearson correlation index

In [119]:
values = pd.DataFrame(ts1.values)
dataframe = pd.concat([values.shift(1), values], axis=1)
dataframe.columns = ['t', 't+1']
result = dataframe.corr()
result
Out[119]:
t t+1
t 1.000000 0.998027
t+1 0.998027 1.000000

Correlation on 99%

This serie is autocorrelated and stationary (no trend or season components). We should be good to go ahead and forecast

Forecasting

We should combine our forecasting algorithm with the characteristics of the serie. Very common choices use to be exponential smoothing methods for non-stationary series and ARIMA for stationary series.

Note: There are many extensions of exponential smoothing methods...to choose the best approach to a given problem, we should again look to the trend and seasonality components. It is also possible to explore RNNs to do forecasting, since this type of Neural Net is used to solve sequence problems.

Here, just because I have a high auto correlated dataset, I will give a try on auto correlation models This is a linear regression model, but the features used are actually your lag variables. AR class has a builtin method to select the optimal number of lag variables as features.

yhat = b0 + (b1 X1) + (b2 X2) ... (bn * Xn), where:

  • b0 is your intercept
  • bn is your coefficient n
  • Xn is your lag variable n
In [120]:
# split the data into train and test (in this case I am selecting the last 25% of the data for testing)
number_points = len(ts1.values)
last_points_for_testing = int(number_points * 0.25)

print ('-------Using {} rows for testing'.format(last_points_for_testing))

data_points = ts1.values
train, test = data_points[1:len(data_points)-last_points_for_testing], data_points[len(data_points)-last_points_for_testing:]

# train AR model
model = AR(train)
model_fit = model.fit()
print('Number of lags: {}'.format(model_fit.k_ar))
print('Coefficients: {}'.format(model_fit.params))

# make predictions of the test period
predictions = model_fit.predict(start=len(train), end=len(train)+len(test)-1, dynamic=False)
for i in range(len(predictions)):
    print('predicted=%f, expected=%f' % (predictions[i], test[i]))

rmse = sqrt(mean_squared_error(test, predictions))
print('Test RMSE: %.3f' % rmse)
-------Using 1250 rows for testing
Number of lags: 30
Coefficients: [ 7.71652707e-14  1.41661465e-02  4.83144820e-02  5.09176075e-01
  2.28157222e-01 -7.38063455e-03 -1.49647474e-01 -2.28980899e-01
 -8.62534344e-02  2.86978215e-01  2.80872941e-01  5.07329404e-02
  1.09498739e-01 -8.44392180e-02 -3.93299758e-02 -2.04488426e-01
  7.11734593e-02  2.78402671e-01  2.36415833e-01  3.24117690e-02
 -1.46836028e-01 -2.12008029e-01 -1.37949266e-01  2.33633220e-02
  8.02837834e-02  1.17627829e-01 -9.13836062e-03 -7.63656246e-02
 -1.43985928e-01 -1.30537591e-01 -7.07925819e-02]
predicted=-0.873738, expected=-0.873736
predicted=-0.902555, expected=-0.902554
predicted=-0.927811, expected=-0.927809
predicted=-0.949405, expected=-0.949402
predicted=-0.967252, expected=-0.967249
predicted=-0.981282, expected=-0.981278
predicted=-0.991439, expected=-0.991435
predicted=-0.997683, expected=-0.997679
predicted=-0.999990, expected=-0.999986
predicted=-0.998351, expected=-0.998346
predicted=-0.992772, expected=-0.992766
predicted=-0.983274, expected=-0.983268
predicted=-0.969896, expected=-0.969890
predicted=-0.952691, expected=-0.952684
predicted=-0.931725, expected=-0.931718
predicted=-0.907083, expected=-0.907075
predicted=-0.878860, expected=-0.878852
predicted=-0.847170, expected=-0.847161
predicted=-0.812136, expected=-0.812127
predicted=-0.773896, expected=-0.773887
predicted=-0.732603, expected=-0.732593
predicted=-0.688419, expected=-0.688408
predicted=-0.641517, expected=-0.641506
predicted=-0.592084, expected=-0.592073
predicted=-0.540314, expected=-0.540302
predicted=-0.486411, expected=-0.486400
predicted=-0.430590, expected=-0.430578
predicted=-0.373068, expected=-0.373056
predicted=-0.314075, expected=-0.314062
predicted=-0.253842, expected=-0.253829
predicted=-0.192607, expected=-0.192594
predicted=-0.130612, expected=-0.130599
predicted=-0.068101, expected=-0.068089
predicted=-0.005322, expected=-0.005310
predicted=0.057479, expected=0.057490
predicted=0.120052, expected=0.120064
predicted=0.182152, expected=0.182163
predicted=0.243533, expected=0.243544
predicted=0.303952, expected=0.303963
predicted=0.363172, expected=0.363183
predicted=0.420959, expected=0.420969
predicted=0.477085, expected=0.477094
predicted=0.531328, expected=0.531336
predicted=0.583474, expected=0.583481
predicted=0.633317, expected=0.633324
predicted=0.680661, expected=0.680667
predicted=0.725319, expected=0.725324
predicted=0.767114, expected=0.767118
predicted=0.805882, expected=0.805885
predicted=0.841469, expected=0.841471
predicted=0.873736, expected=0.873736
predicted=0.902554, expected=0.902554
predicted=0.927810, expected=0.927809
predicted=0.949405, expected=0.949402
predicted=0.967253, expected=0.967249
predicted=0.981283, expected=0.981278
predicted=0.991441, expected=0.991435
predicted=0.997686, expected=0.997679
predicted=0.999994, expected=0.999986
predicted=0.998356, expected=0.998346
predicted=0.992777, expected=0.992766
predicted=0.983280, expected=0.983268
predicted=0.969903, expected=0.969890
predicted=0.952698, expected=0.952684
predicted=0.931733, expected=0.931718
predicted=0.907092, expected=0.907075
predicted=0.878870, expected=0.878852
predicted=0.847180, expected=0.847161
predicted=0.812146, expected=0.812127
predicted=0.773907, expected=0.773887
predicted=0.732614, expected=0.732593
predicted=0.688430, expected=0.688408
predicted=0.641529, expected=0.641506
predicted=0.592096, expected=0.592073
predicted=0.540326, expected=0.540302
predicted=0.486424, expected=0.486400
predicted=0.430602, expected=0.430578
predicted=0.373081, expected=0.373056
predicted=0.314087, expected=0.314062
predicted=0.253854, expected=0.253829
predicted=0.192619, expected=0.192594
predicted=0.130624, expected=0.130599
predicted=0.068113, expected=0.068089
predicted=0.005334, expected=0.005310
predicted=-0.057467, expected=-0.057490
predicted=-0.120041, expected=-0.120064
predicted=-0.182141, expected=-0.182163
predicted=-0.243522, expected=-0.243544
predicted=-0.303942, expected=-0.303963
predicted=-0.363163, expected=-0.363183
predicted=-0.420950, expected=-0.420969
predicted=-0.477077, expected=-0.477094
predicted=-0.531320, expected=-0.531336
predicted=-0.583467, expected=-0.583481
predicted=-0.633311, expected=-0.633324
predicted=-0.680655, expected=-0.680667
predicted=-0.725313, expected=-0.725324
predicted=-0.767109, expected=-0.767118
predicted=-0.805878, expected=-0.805885
predicted=-0.841466, expected=-0.841471
predicted=-0.873733, expected=-0.873736
predicted=-0.902552, expected=-0.902554
predicted=-0.927809, expected=-0.927809
predicted=-0.949405, expected=-0.949402
predicted=-0.967254, expected=-0.967249
predicted=-0.981285, expected=-0.981278
predicted=-0.991444, expected=-0.991435
predicted=-0.997690, expected=-0.997679
predicted=-0.999998, expected=-0.999986
predicted=-0.998361, expected=-0.998346
predicted=-0.992783, expected=-0.992766
predicted=-0.983287, expected=-0.983268
predicted=-0.969910, expected=-0.969890
predicted=-0.952706, expected=-0.952684
predicted=-0.931742, expected=-0.931718
predicted=-0.907100, expected=-0.907075
predicted=-0.878879, expected=-0.878852
predicted=-0.847190, expected=-0.847161
predicted=-0.812156, expected=-0.812127
predicted=-0.773918, expected=-0.773887
predicted=-0.732625, expected=-0.732593
predicted=-0.688442, expected=-0.688408
predicted=-0.641541, expected=-0.641506
predicted=-0.592108, expected=-0.592073
predicted=-0.540338, expected=-0.540302
predicted=-0.486436, expected=-0.486400
predicted=-0.430615, expected=-0.430578
predicted=-0.373094, expected=-0.373056
predicted=-0.314100, expected=-0.314062
predicted=-0.253867, expected=-0.253829
predicted=-0.192632, expected=-0.192594
predicted=-0.130636, expected=-0.130599
predicted=-0.068125, expected=-0.068089
predicted=-0.005346, expected=-0.005310
predicted=0.057455, expected=0.057490
predicted=0.120029, expected=0.120064
predicted=0.182130, expected=0.182163
predicted=0.243511, expected=0.243544
predicted=0.303932, expected=0.303963
predicted=0.363153, expected=0.363183
predicted=0.420941, expected=0.420969
predicted=0.477068, expected=0.477094
predicted=0.531312, expected=0.531336
predicted=0.583459, expected=0.583481
predicted=0.633304, expected=0.633324
predicted=0.680649, expected=0.680667
predicted=0.725308, expected=0.725324
predicted=0.767105, expected=0.767118
predicted=0.805874, expected=0.805885
predicted=0.841463, expected=0.841471
predicted=0.873731, expected=0.873736
predicted=0.902551, expected=0.902554
predicted=0.927809, expected=0.927809
predicted=0.949405, expected=0.949402
predicted=0.967255, expected=0.967249
predicted=0.981287, expected=0.981278
predicted=0.991446, expected=0.991435
predicted=0.997693, expected=0.997679
predicted=1.000002, expected=0.999986
predicted=0.998365, expected=0.998346
predicted=0.992788, expected=0.992766
predicted=0.983293, expected=0.983268
predicted=0.969917, expected=0.969890
predicted=0.952713, expected=0.952684
predicted=0.931750, expected=0.931718
predicted=0.907109, expected=0.907075
predicted=0.878889, expected=0.878852
predicted=0.847199, expected=0.847161
predicted=0.812167, expected=0.812127
predicted=0.773929, expected=0.773887
predicted=0.732637, expected=0.732593
predicted=0.688453, expected=0.688408
predicted=0.641552, expected=0.641506
predicted=0.592120, expected=0.592073
predicted=0.540351, expected=0.540302
predicted=0.486449, expected=0.486400
predicted=0.430627, expected=0.430578
predicted=0.373106, expected=0.373056
predicted=0.314113, expected=0.314062
predicted=0.253879, expected=0.253829
predicted=0.192644, expected=0.192594
predicted=0.130649, expected=0.130599
predicted=0.068138, expected=0.068089
predicted=0.005358, expected=0.005310
predicted=-0.057443, expected=-0.057490
predicted=-0.120018, expected=-0.120064
predicted=-0.182119, expected=-0.182163
predicted=-0.243501, expected=-0.243544
predicted=-0.303922, expected=-0.303963
predicted=-0.363144, expected=-0.363183
predicted=-0.420932, expected=-0.420969
predicted=-0.477059, expected=-0.477094
predicted=-0.531304, expected=-0.531336
predicted=-0.583452, expected=-0.583481
predicted=-0.633297, expected=-0.633324
predicted=-0.680643, expected=-0.680667
predicted=-0.725303, expected=-0.725324
predicted=-0.767100, expected=-0.767118
predicted=-0.805870, expected=-0.805885
predicted=-0.841460, expected=-0.841471
predicted=-0.873729, expected=-0.873736
predicted=-0.902549, expected=-0.902554
predicted=-0.927808, expected=-0.927809
predicted=-0.949405, expected=-0.949402
predicted=-0.967255, expected=-0.967249
predicted=-0.981288, expected=-0.981278
predicted=-0.991449, expected=-0.991435
predicted=-0.997696, expected=-0.997679
predicted=-1.000006, expected=-0.999986
predicted=-0.998370, expected=-0.998346
predicted=-0.992794, expected=-0.992766
predicted=-0.983299, expected=-0.983268
predicted=-0.969924, expected=-0.969890
predicted=-0.952721, expected=-0.952684
predicted=-0.931758, expected=-0.931718
predicted=-0.907118, expected=-0.907075
predicted=-0.878898, expected=-0.878852
predicted=-0.847209, expected=-0.847161
predicted=-0.812177, expected=-0.812127
predicted=-0.773940, expected=-0.773887
predicted=-0.732648, expected=-0.732593
predicted=-0.688465, expected=-0.688408
predicted=-0.641564, expected=-0.641506
predicted=-0.592132, expected=-0.592073
predicted=-0.540363, expected=-0.540302
predicted=-0.486461, expected=-0.486400
predicted=-0.430640, expected=-0.430578
predicted=-0.373119, expected=-0.373056
predicted=-0.314125, expected=-0.314062
predicted=-0.253892, expected=-0.253829
predicted=-0.192657, expected=-0.192594
predicted=-0.130661, expected=-0.130599
predicted=-0.068150, expected=-0.068089
predicted=-0.005370, expected=-0.005310
predicted=0.057432, expected=0.057490
predicted=0.120006, expected=0.120064
predicted=0.182108, expected=0.182163
predicted=0.243490, expected=0.243544
predicted=0.303912, expected=0.303963
predicted=0.363134, expected=0.363183
predicted=0.420923, expected=0.420969
predicted=0.477051, expected=0.477094
predicted=0.531296, expected=0.531336
predicted=0.583445, expected=0.583481
predicted=0.633290, expected=0.633324
predicted=0.680637, expected=0.680667
predicted=0.725298, expected=0.725324
predicted=0.767096, expected=0.767118
predicted=0.805866, expected=0.805885
predicted=0.841457, expected=0.841471
predicted=0.873726, expected=0.873736
predicted=0.902548, expected=0.902554
predicted=0.927807, expected=0.927809
predicted=0.949405, expected=0.949402
predicted=0.967256, expected=0.967249
predicted=0.981290, expected=0.981278
predicted=0.991451, expected=0.991435
predicted=0.997700, expected=0.997679
predicted=1.000010, expected=0.999986
predicted=0.998375, expected=0.998346
predicted=0.992799, expected=0.992766
predicted=0.983305, expected=0.983268
predicted=0.969931, expected=0.969890
predicted=0.952728, expected=0.952684
predicted=0.931766, expected=0.931718
predicted=0.907127, expected=0.907075
predicted=0.878907, expected=0.878852
predicted=0.847219, expected=0.847161
predicted=0.812187, expected=0.812127
predicted=0.773950, expected=0.773887
predicted=0.732659, expected=0.732593
predicted=0.688476, expected=0.688408
predicted=0.641576, expected=0.641506
predicted=0.592144, expected=0.592073
predicted=0.540375, expected=0.540302
predicted=0.486474, expected=0.486400
predicted=0.430652, expected=0.430578
predicted=0.373131, expected=0.373056
predicted=0.314138, expected=0.314062
predicted=0.253905, expected=0.253829
predicted=0.192669, expected=0.192594
predicted=0.130674, expected=0.130599
predicted=0.068162, expected=0.068089
predicted=0.005382, expected=0.005310
predicted=-0.057420, expected=-0.057490
predicted=-0.119995, expected=-0.120064
predicted=-0.182097, expected=-0.182163
predicted=-0.243479, expected=-0.243544
predicted=-0.303901, expected=-0.303963
predicted=-0.363124, expected=-0.363183
predicted=-0.420914, expected=-0.420969
predicted=-0.477042, expected=-0.477094
predicted=-0.531288, expected=-0.531336
predicted=-0.583437, expected=-0.583481
predicted=-0.633284, expected=-0.633324
predicted=-0.680631, expected=-0.680667
predicted=-0.725292, expected=-0.725324
predicted=-0.767091, expected=-0.767118
predicted=-0.805863, expected=-0.805885
predicted=-0.841454, expected=-0.841471
predicted=-0.873724, expected=-0.873736
predicted=-0.902546, expected=-0.902554
predicted=-0.927807, expected=-0.927809
predicted=-0.949405, expected=-0.949402
predicted=-0.967257, expected=-0.967249
predicted=-0.981292, expected=-0.981278
predicted=-0.991454, expected=-0.991435
predicted=-0.997703, expected=-0.997679
predicted=-1.000015, expected=-0.999986
predicted=-0.998380, expected=-0.998346
predicted=-0.992805, expected=-0.992766
predicted=-0.983311, expected=-0.983268
predicted=-0.969938, expected=-0.969890
predicted=-0.952736, expected=-0.952684
predicted=-0.931774, expected=-0.931718
predicted=-0.907135, expected=-0.907075
predicted=-0.878916, expected=-0.878852
predicted=-0.847229, expected=-0.847161
predicted=-0.812198, expected=-0.812127
predicted=-0.773961, expected=-0.773887
predicted=-0.732670, expected=-0.732593
predicted=-0.688488, expected=-0.688408
predicted=-0.641588, expected=-0.641506
predicted=-0.592156, expected=-0.592073
predicted=-0.540388, expected=-0.540302
predicted=-0.486486, expected=-0.486400
predicted=-0.430665, expected=-0.430578
predicted=-0.373144, expected=-0.373056
predicted=-0.314151, expected=-0.314062
predicted=-0.253917, expected=-0.253829
predicted=-0.192682, expected=-0.192594
predicted=-0.130686, expected=-0.130599
predicted=-0.068174, expected=-0.068089
predicted=-0.005394, expected=-0.005310
predicted=0.057408, expected=0.057490
predicted=0.119984, expected=0.120064
predicted=0.182086, expected=0.182163
predicted=0.243469, expected=0.243544
predicted=0.303891, expected=0.303963
predicted=0.363114, expected=0.363183
predicted=0.420905, expected=0.420969
predicted=0.477034, expected=0.477094
predicted=0.531280, expected=0.531336
predicted=0.583430, expected=0.583481
predicted=0.633277, expected=0.633324
predicted=0.680625, expected=0.680667
predicted=0.725287, expected=0.725324
predicted=0.767087, expected=0.767118
predicted=0.805859, expected=0.805885
predicted=0.841451, expected=0.841471
predicted=0.873722, expected=0.873736
predicted=0.902545, expected=0.902554
predicted=0.927806, expected=0.927809
predicted=0.949406, expected=0.949402
predicted=0.967258, expected=0.967249
predicted=0.981294, expected=0.981278
predicted=0.991456, expected=0.991435
predicted=0.997706, expected=0.997679
predicted=1.000019, expected=0.999986
predicted=0.998384, expected=0.998346
predicted=0.992810, expected=0.992766
predicted=0.983318, expected=0.983268
predicted=0.969945, expected=0.969890
predicted=0.952744, expected=0.952684
predicted=0.931782, expected=0.931718
predicted=0.907144, expected=0.907075
predicted=0.878926, expected=0.878852
predicted=0.847239, expected=0.847161
predicted=0.812208, expected=0.812127
predicted=0.773972, expected=0.773887
predicted=0.732681, expected=0.732593
predicted=0.688499, expected=0.688408
predicted=0.641600, expected=0.641506
predicted=0.592168, expected=0.592073
predicted=0.540400, expected=0.540302
predicted=0.486499, expected=0.486400
predicted=0.430678, expected=0.430578
predicted=0.373157, expected=0.373056
predicted=0.314163, expected=0.314062
predicted=0.253930, expected=0.253829
predicted=0.192694, expected=0.192594
predicted=0.130698, expected=0.130599
predicted=0.068187, expected=0.068089
predicted=0.005406, expected=0.005310
predicted=-0.057396, expected=-0.057490
predicted=-0.119972, expected=-0.120064
predicted=-0.182074, expected=-0.182163
predicted=-0.243458, expected=-0.243544
predicted=-0.303881, expected=-0.303963
predicted=-0.363105, expected=-0.363183
predicted=-0.420895, expected=-0.420969
predicted=-0.477025, expected=-0.477094
predicted=-0.531272, expected=-0.531336
predicted=-0.583423, expected=-0.583481
predicted=-0.633270, expected=-0.633324
predicted=-0.680619, expected=-0.680667
predicted=-0.725282, expected=-0.725324
predicted=-0.767082, expected=-0.767118
predicted=-0.805855, expected=-0.805885
predicted=-0.841448, expected=-0.841471
predicted=-0.873720, expected=-0.873736
predicted=-0.902543, expected=-0.902554
predicted=-0.927805, expected=-0.927809
predicted=-0.949406, expected=-0.949402
predicted=-0.967259, expected=-0.967249
predicted=-0.981295, expected=-0.981278
predicted=-0.991459, expected=-0.991435
predicted=-0.997709, expected=-0.997679
predicted=-1.000023, expected=-0.999986
predicted=-0.998389, expected=-0.998346
predicted=-0.992816, expected=-0.992766
predicted=-0.983324, expected=-0.983268
predicted=-0.969951, expected=-0.969890
predicted=-0.952751, expected=-0.952684
predicted=-0.931791, expected=-0.931718
predicted=-0.907153, expected=-0.907075
predicted=-0.878935, expected=-0.878852
predicted=-0.847249, expected=-0.847161
predicted=-0.812218, expected=-0.812127
predicted=-0.773983, expected=-0.773887
predicted=-0.732692, expected=-0.732593
predicted=-0.688511, expected=-0.688408
predicted=-0.641612, expected=-0.641506
predicted=-0.592180, expected=-0.592073
predicted=-0.540412, expected=-0.540302
predicted=-0.486511, expected=-0.486400
predicted=-0.430690, expected=-0.430578
predicted=-0.373169, expected=-0.373056
predicted=-0.314176, expected=-0.314062
predicted=-0.253943, expected=-0.253829
predicted=-0.192707, expected=-0.192594
predicted=-0.130711, expected=-0.130599
predicted=-0.068199, expected=-0.068089
predicted=-0.005418, expected=-0.005310
predicted=0.057385, expected=0.057490
predicted=0.119961, expected=0.120064
predicted=0.182063, expected=0.182163
predicted=0.243448, expected=0.243544
predicted=0.303871, expected=0.303963
predicted=0.363095, expected=0.363183
predicted=0.420886, expected=0.420969
predicted=0.477017, expected=0.477094
predicted=0.531264, expected=0.531336
predicted=0.583415, expected=0.583481
predicted=0.633264, expected=0.633324
predicted=0.680613, expected=0.680667
predicted=0.725276, expected=0.725324
predicted=0.767077, expected=0.767118
predicted=0.805851, expected=0.805885
predicted=0.841445, expected=0.841471
predicted=0.873717, expected=0.873736
predicted=0.902542, expected=0.902554
predicted=0.927805, expected=0.927809
predicted=0.949406, expected=0.949402
predicted=0.967260, expected=0.967249
predicted=0.981297, expected=0.981278
predicted=0.991461, expected=0.991435
predicted=0.997713, expected=0.997679
predicted=1.000027, expected=0.999986
predicted=0.998394, expected=0.998346
predicted=0.992821, expected=0.992766
predicted=0.983330, expected=0.983268
predicted=0.969958, expected=0.969890
predicted=0.952759, expected=0.952684
predicted=0.931799, expected=0.931718
predicted=0.907162, expected=0.907075
predicted=0.878944, expected=0.878852
predicted=0.847258, expected=0.847161
predicted=0.812229, expected=0.812127
predicted=0.773993, expected=0.773887
predicted=0.732704, expected=0.732593
predicted=0.688522, expected=0.688408
predicted=0.641623, expected=0.641506
predicted=0.592192, expected=0.592073
predicted=0.540424, expected=0.540302
predicted=0.486524, expected=0.486400
predicted=0.430703, expected=0.430578
predicted=0.373182, expected=0.373056
predicted=0.314189, expected=0.314062
predicted=0.253955, expected=0.253829
predicted=0.192720, expected=0.192594
predicted=0.130723, expected=0.130599
predicted=0.068211, expected=0.068089
predicted=0.005430, expected=0.005310
predicted=-0.057373, expected=-0.057490
predicted=-0.119949, expected=-0.120064
predicted=-0.182052, expected=-0.182163
predicted=-0.243437, expected=-0.243544
predicted=-0.303861, expected=-0.303963
predicted=-0.363085, expected=-0.363183
predicted=-0.420877, expected=-0.420969
predicted=-0.477008, expected=-0.477094
predicted=-0.531256, expected=-0.531336
predicted=-0.583408, expected=-0.583481
predicted=-0.633257, expected=-0.633324
predicted=-0.680607, expected=-0.680667
predicted=-0.725271, expected=-0.725324
predicted=-0.767073, expected=-0.767118
predicted=-0.805847, expected=-0.805885
predicted=-0.841442, expected=-0.841471
predicted=-0.873715, expected=-0.873736
predicted=-0.902541, expected=-0.902554
predicted=-0.927804, expected=-0.927809
predicted=-0.949406, expected=-0.949402
predicted=-0.967261, expected=-0.967249
predicted=-0.981299, expected=-0.981278
predicted=-0.991464, expected=-0.991435
predicted=-0.997716, expected=-0.997679
predicted=-1.000031, expected=-0.999986
predicted=-0.998399, expected=-0.998346
predicted=-0.992827, expected=-0.992766
predicted=-0.983336, expected=-0.983268
predicted=-0.969965, expected=-0.969890
predicted=-0.952766, expected=-0.952684
predicted=-0.931807, expected=-0.931718
predicted=-0.907170, expected=-0.907075
predicted=-0.878954, expected=-0.878852
predicted=-0.847268, expected=-0.847161
predicted=-0.812239, expected=-0.812127
predicted=-0.774004, expected=-0.773887
predicted=-0.732715, expected=-0.732593
predicted=-0.688534, expected=-0.688408
predicted=-0.641635, expected=-0.641506
predicted=-0.592205, expected=-0.592073
predicted=-0.540437, expected=-0.540302
predicted=-0.486536, expected=-0.486400
predicted=-0.430715, expected=-0.430578
predicted=-0.373195, expected=-0.373056
predicted=-0.314201, expected=-0.314062
predicted=-0.253968, expected=-0.253829
predicted=-0.192732, expected=-0.192594
predicted=-0.130736, expected=-0.130599
predicted=-0.068223, expected=-0.068089
predicted=-0.005442, expected=-0.005310
predicted=0.057361, expected=0.057490
predicted=0.119938, expected=0.120064
predicted=0.182041, expected=0.182163
predicted=0.243426, expected=0.243544
predicted=0.303851, expected=0.303963
predicted=0.363076, expected=0.363183
predicted=0.420868, expected=0.420969
predicted=0.476999, expected=0.477094
predicted=0.531248, expected=0.531336
predicted=0.583400, expected=0.583481
predicted=0.633250, expected=0.633324
predicted=0.680601, expected=0.680667
predicted=0.725266, expected=0.725324
predicted=0.767068, expected=0.767118
predicted=0.805844, expected=0.805885
predicted=0.841439, expected=0.841471
predicted=0.873713, expected=0.873736
predicted=0.902539, expected=0.902554
predicted=0.927803, expected=0.927809
predicted=0.949406, expected=0.949402
predicted=0.967262, expected=0.967249
predicted=0.981300, expected=0.981278
predicted=0.991466, expected=0.991435
predicted=0.997719, expected=0.997679
predicted=1.000035, expected=0.999986
predicted=0.998403, expected=0.998346
predicted=0.992832, expected=0.992766
predicted=0.983342, expected=0.983268
predicted=0.969972, expected=0.969890
predicted=0.952774, expected=0.952684
predicted=0.931815, expected=0.931718
predicted=0.907179, expected=0.907075
predicted=0.878963, expected=0.878852
predicted=0.847278, expected=0.847161
predicted=0.812249, expected=0.812127
predicted=0.774015, expected=0.773887
predicted=0.732726, expected=0.732593
predicted=0.688545, expected=0.688408
predicted=0.641647, expected=0.641506
predicted=0.592217, expected=0.592073
predicted=0.540449, expected=0.540302
predicted=0.486549, expected=0.486400
predicted=0.430728, expected=0.430578
predicted=0.373207, expected=0.373056
predicted=0.314214, expected=0.314062
predicted=0.253980, expected=0.253829
predicted=0.192745, expected=0.192594
predicted=0.130748, expected=0.130599
predicted=0.068236, expected=0.068089
predicted=0.005454, expected=0.005310
predicted=-0.057350, expected=-0.057490
predicted=-0.119927, expected=-0.120064
predicted=-0.182030, expected=-0.182163
predicted=-0.243416, expected=-0.243544
predicted=-0.303840, expected=-0.303963
predicted=-0.363066, expected=-0.363183
predicted=-0.420859, expected=-0.420969
predicted=-0.476991, expected=-0.477094
predicted=-0.531240, expected=-0.531336
predicted=-0.583393, expected=-0.583481
predicted=-0.633244, expected=-0.633324
predicted=-0.680595, expected=-0.680667
predicted=-0.725261, expected=-0.725324
predicted=-0.767064, expected=-0.767118
predicted=-0.805840, expected=-0.805885
predicted=-0.841436, expected=-0.841471
predicted=-0.873711, expected=-0.873736
predicted=-0.902538, expected=-0.902554
predicted=-0.927803, expected=-0.927809
predicted=-0.949406, expected=-0.949402
predicted=-0.967263, expected=-0.967249
predicted=-0.981302, expected=-0.981278
predicted=-0.991469, expected=-0.991435
predicted=-0.997722, expected=-0.997679
predicted=-1.000039, expected=-0.999986
predicted=-0.998408, expected=-0.998346
predicted=-0.992838, expected=-0.992766
predicted=-0.983349, expected=-0.983268
predicted=-0.969979, expected=-0.969890
predicted=-0.952781, expected=-0.952684
predicted=-0.931823, expected=-0.931718
predicted=-0.907188, expected=-0.907075
predicted=-0.878972, expected=-0.878852
predicted=-0.847288, expected=-0.847161
predicted=-0.812260, expected=-0.812127
predicted=-0.774026, expected=-0.773887
predicted=-0.732737, expected=-0.732593
predicted=-0.688557, expected=-0.688408
predicted=-0.641659, expected=-0.641506
predicted=-0.592229, expected=-0.592073
predicted=-0.540461, expected=-0.540302
predicted=-0.486561, expected=-0.486400
predicted=-0.430740, expected=-0.430578
predicted=-0.373220, expected=-0.373056
predicted=-0.314227, expected=-0.314062
predicted=-0.253993, expected=-0.253829
predicted=-0.192757, expected=-0.192594
predicted=-0.130761, expected=-0.130599
predicted=-0.068248, expected=-0.068089
predicted=-0.005466, expected=-0.005310
predicted=0.057338, expected=0.057490
predicted=0.119915, expected=0.120064
predicted=0.182019, expected=0.182163
predicted=0.243405, expected=0.243544
predicted=0.303830, expected=0.303963
predicted=0.363056, expected=0.363183
predicted=0.420850, expected=0.420969
predicted=0.476982, expected=0.477094
predicted=0.531232, expected=0.531336
predicted=0.583386, expected=0.583481
predicted=0.633237, expected=0.633324
predicted=0.680589, expected=0.680667
predicted=0.725255, expected=0.725324
predicted=0.767059, expected=0.767118
predicted=0.805836, expected=0.805885
predicted=0.841433, expected=0.841471
predicted=0.873708, expected=0.873736
predicted=0.902536, expected=0.902554
predicted=0.927802, expected=0.927809
predicted=0.949406, expected=0.949402
predicted=0.967264, expected=0.967249
predicted=0.981304, expected=0.981278
predicted=0.991471, expected=0.991435
predicted=0.997726, expected=0.997679
predicted=1.000043, expected=0.999986
predicted=0.998413, expected=0.998346
predicted=0.992843, expected=0.992766
predicted=0.983355, expected=0.983268
predicted=0.969986, expected=0.969890
predicted=0.952789, expected=0.952684
predicted=0.931831, expected=0.931718
predicted=0.907197, expected=0.907075
predicted=0.878982, expected=0.878852
predicted=0.847298, expected=0.847161
predicted=0.812270, expected=0.812127
predicted=0.774036, expected=0.773887
predicted=0.732748, expected=0.732593
predicted=0.688568, expected=0.688408
predicted=0.641671, expected=0.641506
predicted=0.592241, expected=0.592073
predicted=0.540474, expected=0.540302
predicted=0.486573, expected=0.486400
predicted=0.430753, expected=0.430578
predicted=0.373233, expected=0.373056
predicted=0.314239, expected=0.314062
predicted=0.254006, expected=0.253829
predicted=0.192770, expected=0.192594
predicted=0.130773, expected=0.130599
predicted=0.068260, expected=0.068089
predicted=0.005478, expected=0.005310
predicted=-0.057326, expected=-0.057490
predicted=-0.119904, expected=-0.120064
predicted=-0.182008, expected=-0.182163
predicted=-0.243394, expected=-0.243544
predicted=-0.303820, expected=-0.303963
predicted=-0.363047, expected=-0.363183
predicted=-0.420841, expected=-0.420969
predicted=-0.476974, expected=-0.477094
predicted=-0.531224, expected=-0.531336
predicted=-0.583378, expected=-0.583481
predicted=-0.633230, expected=-0.633324
predicted=-0.680583, expected=-0.680667
predicted=-0.725250, expected=-0.725324
predicted=-0.767055, expected=-0.767118
predicted=-0.805832, expected=-0.805885
predicted=-0.841430, expected=-0.841471
predicted=-0.873706, expected=-0.873736
predicted=-0.902535, expected=-0.902554
predicted=-0.927801, expected=-0.927809
predicted=-0.949406, expected=-0.949402
predicted=-0.967265, expected=-0.967249
predicted=-0.981305, expected=-0.981278
predicted=-0.991474, expected=-0.991435
predicted=-0.997729, expected=-0.997679
predicted=-1.000047, expected=-0.999986
predicted=-0.998418, expected=-0.998346
predicted=-0.992849, expected=-0.992766
predicted=-0.983361, expected=-0.983268
predicted=-0.969993, expected=-0.969890
predicted=-0.952796, expected=-0.952684
predicted=-0.931840, expected=-0.931718
predicted=-0.907205, expected=-0.907075
predicted=-0.878991, expected=-0.878852
predicted=-0.847308, expected=-0.847161
predicted=-0.812280, expected=-0.812127
predicted=-0.774047, expected=-0.773887
predicted=-0.732759, expected=-0.732593
predicted=-0.688580, expected=-0.688408
predicted=-0.641682, expected=-0.641506
predicted=-0.592253, expected=-0.592073
predicted=-0.540486, expected=-0.540302
predicted=-0.486586, expected=-0.486400
predicted=-0.430766, expected=-0.430578
predicted=-0.373245, expected=-0.373056
predicted=-0.314252, expected=-0.314062
predicted=-0.254018, expected=-0.253829
predicted=-0.192782, expected=-0.192594
predicted=-0.130785, expected=-0.130599
predicted=-0.068272, expected=-0.068089
predicted=-0.005490, expected=-0.005310
predicted=0.057314, expected=0.057490
predicted=0.119892, expected=0.120064
predicted=0.181997, expected=0.182163
predicted=0.243384, expected=0.243544
predicted=0.303810, expected=0.303963
predicted=0.363037, expected=0.363183
predicted=0.420831, expected=0.420969
predicted=0.476965, expected=0.477094
predicted=0.531216, expected=0.531336
predicted=0.583371, expected=0.583481
predicted=0.633224, expected=0.633324
predicted=0.680577, expected=0.680667
predicted=0.725245, expected=0.725324
predicted=0.767050, expected=0.767118
predicted=0.805828, expected=0.805885
predicted=0.841427, expected=0.841471
predicted=0.873704, expected=0.873736
predicted=0.902533, expected=0.902554
predicted=0.927801, expected=0.927809
predicted=0.949406, expected=0.949402
predicted=0.967265, expected=0.967249
predicted=0.981307, expected=0.981278
predicted=0.991476, expected=0.991435
predicted=0.997732, expected=0.997679
predicted=1.000051, expected=0.999986
predicted=0.998423, expected=0.998346
predicted=0.992854, expected=0.992766
predicted=0.983367, expected=0.983268
predicted=0.970000, expected=0.969890
predicted=0.952804, expected=0.952684
predicted=0.931848, expected=0.931718
predicted=0.907214, expected=0.907075
predicted=0.879000, expected=0.878852
predicted=0.847317, expected=0.847161
predicted=0.812290, expected=0.812127
predicted=0.774058, expected=0.773887
predicted=0.732770, expected=0.732593
predicted=0.688591, expected=0.688408
predicted=0.641694, expected=0.641506
predicted=0.592265, expected=0.592073
predicted=0.540498, expected=0.540302
predicted=0.486598, expected=0.486400
predicted=0.430778, expected=0.430578
predicted=0.373258, expected=0.373056
predicted=0.314265, expected=0.314062
predicted=0.254031, expected=0.253829
predicted=0.192795, expected=0.192594
predicted=0.130798, expected=0.130599
predicted=0.068285, expected=0.068089
predicted=0.005502, expected=0.005310
predicted=-0.057303, expected=-0.057490
predicted=-0.119881, expected=-0.120064
predicted=-0.181986, expected=-0.182163
predicted=-0.243373, expected=-0.243544
predicted=-0.303800, expected=-0.303963
predicted=-0.363027, expected=-0.363183
predicted=-0.420822, expected=-0.420969
predicted=-0.476956, expected=-0.477094
predicted=-0.531208, expected=-0.531336
predicted=-0.583364, expected=-0.583481
predicted=-0.633217, expected=-0.633324
predicted=-0.680571, expected=-0.680667
predicted=-0.725239, expected=-0.725324
predicted=-0.767046, expected=-0.767118
predicted=-0.805825, expected=-0.805885
predicted=-0.841423, expected=-0.841471
predicted=-0.873702, expected=-0.873736
predicted=-0.902532, expected=-0.902554
predicted=-0.927800, expected=-0.927809
predicted=-0.949407, expected=-0.949402
predicted=-0.967266, expected=-0.967249
predicted=-0.981309, expected=-0.981278
predicted=-0.991479, expected=-0.991435
predicted=-0.997735, expected=-0.997679
predicted=-1.000055, expected=-0.999986
predicted=-0.998427, expected=-0.998346
predicted=-0.992860, expected=-0.992766
predicted=-0.983373, expected=-0.983268
predicted=-0.970006, expected=-0.969890
predicted=-0.952811, expected=-0.952684
predicted=-0.931856, expected=-0.931718
predicted=-0.907223, expected=-0.907075
predicted=-0.879010, expected=-0.878852
predicted=-0.847327, expected=-0.847161
predicted=-0.812301, expected=-0.812127
predicted=-0.774069, expected=-0.773887
predicted=-0.732782, expected=-0.732593
predicted=-0.688603, expected=-0.688408
predicted=-0.641706, expected=-0.641506
predicted=-0.592277, expected=-0.592073
predicted=-0.540510, expected=-0.540302
predicted=-0.486611, expected=-0.486400
predicted=-0.430791, expected=-0.430578
predicted=-0.373271, expected=-0.373056
predicted=-0.314277, expected=-0.314062
predicted=-0.254044, expected=-0.253829
predicted=-0.192807, expected=-0.192594
predicted=-0.130810, expected=-0.130599
predicted=-0.068297, expected=-0.068089
predicted=-0.005514, expected=-0.005310
predicted=0.057291, expected=0.057490
predicted=0.119870, expected=0.120064
predicted=0.181975, expected=0.182163
predicted=0.243363, expected=0.243544
predicted=0.303790, expected=0.303963
predicted=0.363018, expected=0.363183
predicted=0.420813, expected=0.420969
predicted=0.476948, expected=0.477094
predicted=0.531200, expected=0.531336
predicted=0.583356, expected=0.583481
predicted=0.633210, expected=0.633324
predicted=0.680565, expected=0.680667
predicted=0.725234, expected=0.725324
predicted=0.767041, expected=0.767118
predicted=0.805821, expected=0.805885
predicted=0.841420, expected=0.841471
predicted=0.873699, expected=0.873736
predicted=0.902530, expected=0.902554
predicted=0.927799, expected=0.927809
predicted=0.949407, expected=0.949402
predicted=0.967267, expected=0.967249
predicted=0.981311, expected=0.981278
predicted=0.991481, expected=0.991435
predicted=0.997739, expected=0.997679
predicted=1.000059, expected=0.999986
predicted=0.998432, expected=0.998346
predicted=0.992865, expected=0.992766
predicted=0.983380, expected=0.983268
predicted=0.970013, expected=0.969890
predicted=0.952819, expected=0.952684
predicted=0.931864, expected=0.931718
predicted=0.907232, expected=0.907075
predicted=0.879019, expected=0.878852
predicted=0.847337, expected=0.847161
predicted=0.812311, expected=0.812127
predicted=0.774079, expected=0.773887
predicted=0.732793, expected=0.732593
predicted=0.688614, expected=0.688408
predicted=0.641718, expected=0.641506
predicted=0.592289, expected=0.592073
predicted=0.540523, expected=0.540302
predicted=0.486623, expected=0.486400
predicted=0.430803, expected=0.430578
predicted=0.373283, expected=0.373056
predicted=0.314290, expected=0.314062
predicted=0.254056, expected=0.253829
predicted=0.192820, expected=0.192594
predicted=0.130823, expected=0.130599
predicted=0.068309, expected=0.068089
predicted=0.005526, expected=0.005310
predicted=-0.057279, expected=-0.057490
predicted=-0.119858, expected=-0.120064
predicted=-0.181964, expected=-0.182163
predicted=-0.243352, expected=-0.243544
predicted=-0.303779, expected=-0.303963
predicted=-0.363008, expected=-0.363183
predicted=-0.420804, expected=-0.420969
predicted=-0.476939, expected=-0.477094
predicted=-0.531192, expected=-0.531336
predicted=-0.583349, expected=-0.583481
predicted=-0.633204, expected=-0.633324
predicted=-0.680559, expected=-0.680667
predicted=-0.725229, expected=-0.725324
predicted=-0.767037, expected=-0.767118
predicted=-0.805817, expected=-0.805885
predicted=-0.841417, expected=-0.841471
predicted=-0.873697, expected=-0.873736
predicted=-0.902529, expected=-0.902554
predicted=-0.927799, expected=-0.927809
predicted=-0.949407, expected=-0.949402
predicted=-0.967268, expected=-0.967249
predicted=-0.981312, expected=-0.981278
predicted=-0.991484, expected=-0.991435
predicted=-0.997742, expected=-0.997679
predicted=-1.000063, expected=-0.999986
predicted=-0.998437, expected=-0.998346
predicted=-0.992871, expected=-0.992766
predicted=-0.983386, expected=-0.983268
predicted=-0.970020, expected=-0.969890
predicted=-0.952826, expected=-0.952684
predicted=-0.931872, expected=-0.931718
predicted=-0.907240, expected=-0.907075
predicted=-0.879028, expected=-0.878852
predicted=-0.847347, expected=-0.847161
predicted=-0.812321, expected=-0.812127
predicted=-0.774090, expected=-0.773887
predicted=-0.732804, expected=-0.732593
predicted=-0.688626, expected=-0.688408
predicted=-0.641730, expected=-0.641506
predicted=-0.592301, expected=-0.592073
predicted=-0.540535, expected=-0.540302
predicted=-0.486636, expected=-0.486400
predicted=-0.430816, expected=-0.430578
predicted=-0.373296, expected=-0.373056
predicted=-0.314303, expected=-0.314062
predicted=-0.254069, expected=-0.253829
predicted=-0.192832, expected=-0.192594
predicted=-0.130835, expected=-0.130599
predicted=-0.068321, expected=-0.068089
predicted=-0.005538, expected=-0.005310
predicted=0.057267, expected=0.057490
predicted=0.119847, expected=0.120064
predicted=0.181953, expected=0.182163
predicted=0.243341, expected=0.243544
predicted=0.303769, expected=0.303963
predicted=0.362998, expected=0.363183
predicted=0.420795, expected=0.420969
predicted=0.476931, expected=0.477094
predicted=0.531184, expected=0.531336
predicted=0.583342, expected=0.583481
predicted=0.633197, expected=0.633324
predicted=0.680553, expected=0.680667
predicted=0.725224, expected=0.725324
predicted=0.767032, expected=0.767118
predicted=0.805813, expected=0.805885
predicted=0.841414, expected=0.841471
predicted=0.873695, expected=0.873736
predicted=0.902527, expected=0.902554
predicted=0.927798, expected=0.927809
predicted=0.949407, expected=0.949402
predicted=0.967269, expected=0.967249
predicted=0.981314, expected=0.981278
predicted=0.991486, expected=0.991435
predicted=0.997745, expected=0.997679
predicted=1.000067, expected=0.999986
predicted=0.998442, expected=0.998346
predicted=0.992876, expected=0.992766
predicted=0.983392, expected=0.983268
predicted=0.970027, expected=0.969890
predicted=0.952834, expected=0.952684
predicted=0.931880, expected=0.931718
predicted=0.907249, expected=0.907075
predicted=0.879038, expected=0.878852
predicted=0.847357, expected=0.847161
predicted=0.812332, expected=0.812127
predicted=0.774101, expected=0.773887
predicted=0.732815, expected=0.732593
predicted=0.688637, expected=0.688408
predicted=0.641742, expected=0.641506
predicted=0.592313, expected=0.592073
predicted=0.540547, expected=0.540302
predicted=0.486648, expected=0.486400
predicted=0.430828, expected=0.430578
predicted=0.373308, expected=0.373056
predicted=0.314315, expected=0.314062
predicted=0.254081, expected=0.253829
predicted=0.192845, expected=0.192594
predicted=0.130847, expected=0.130599
predicted=0.068333, expected=0.068089
predicted=0.005550, expected=0.005310
predicted=-0.057256, expected=-0.057490
predicted=-0.119835, expected=-0.120064
predicted=-0.181942, expected=-0.182163
predicted=-0.243331, expected=-0.243544
predicted=-0.303759, expected=-0.303963
predicted=-0.362989, expected=-0.363183
predicted=-0.420786, expected=-0.420969
predicted=-0.476922, expected=-0.477094
predicted=-0.531176, expected=-0.531336
predicted=-0.583334, expected=-0.583481
predicted=-0.633190, expected=-0.633324
predicted=-0.680547, expected=-0.680667
predicted=-0.725218, expected=-0.725324
predicted=-0.767027, expected=-0.767118
predicted=-0.805809, expected=-0.805885
predicted=-0.841411, expected=-0.841471
predicted=-0.873693, expected=-0.873736
predicted=-0.902526, expected=-0.902554
predicted=-0.927797, expected=-0.927809
predicted=-0.949407, expected=-0.949402
predicted=-0.967270, expected=-0.967249
predicted=-0.981316, expected=-0.981278
predicted=-0.991489, expected=-0.991435
predicted=-0.997749, expected=-0.997679
predicted=-1.000071, expected=-0.999986
predicted=-0.998446, expected=-0.998346
predicted=-0.992882, expected=-0.992766
predicted=-0.983398, expected=-0.983268
predicted=-0.970034, expected=-0.969890
predicted=-0.952842, expected=-0.952684
predicted=-0.931889, expected=-0.931718
predicted=-0.907258, expected=-0.907075
predicted=-0.879047, expected=-0.878852
predicted=-0.847366, expected=-0.847161
predicted=-0.812342, expected=-0.812127
predicted=-0.774112, expected=-0.773887
predicted=-0.732826, expected=-0.732593
predicted=-0.688649, expected=-0.688408
predicted=-0.641753, expected=-0.641506
predicted=-0.592325, expected=-0.592073
predicted=-0.540560, expected=-0.540302
predicted=-0.486661, expected=-0.486400
predicted=-0.430841, expected=-0.430578
predicted=-0.373321, expected=-0.373056
predicted=-0.314328, expected=-0.314062
predicted=-0.254094, expected=-0.253829
predicted=-0.192858, expected=-0.192594
predicted=-0.130860, expected=-0.130599
predicted=-0.068346, expected=-0.068089
predicted=-0.005562, expected=-0.005310
predicted=0.057244, expected=0.057490
predicted=0.119824, expected=0.120064
predicted=0.181931, expected=0.182163
predicted=0.243320, expected=0.243544
predicted=0.303749, expected=0.303963
predicted=0.362979, expected=0.363183
predicted=0.420777, expected=0.420969
predicted=0.476914, expected=0.477094
predicted=0.531168, expected=0.531336
predicted=0.583327, expected=0.583481
predicted=0.633184, expected=0.633324
predicted=0.680541, expected=0.680667
predicted=0.725213, expected=0.725324
predicted=0.767023, expected=0.767118
predicted=0.805806, expected=0.805885
predicted=0.841408, expected=0.841471
predicted=0.873690, expected=0.873736
predicted=0.902524, expected=0.902554
predicted=0.927797, expected=0.927809
predicted=0.949407, expected=0.949402
predicted=0.967271, expected=0.967249
predicted=0.981317, expected=0.981278
predicted=0.991491, expected=0.991435
predicted=0.997752, expected=0.997679
predicted=1.000075, expected=0.999986
predicted=0.998451, expected=0.998346
predicted=0.992887, expected=0.992766
predicted=0.983405, expected=0.983268
predicted=0.970041, expected=0.969890
predicted=0.952849, expected=0.952684
predicted=0.931897, expected=0.931718
predicted=0.907267, expected=0.907075
predicted=0.879056, expected=0.878852
predicted=0.847376, expected=0.847161
predicted=0.812352, expected=0.812127
predicted=0.774122, expected=0.773887
predicted=0.732837, expected=0.732593
predicted=0.688660, expected=0.688408
predicted=0.641765, expected=0.641506
predicted=0.592337, expected=0.592073
predicted=0.540572, expected=0.540302
predicted=0.486673, expected=0.486400
predicted=0.430854, expected=0.430578
predicted=0.373334, expected=0.373056
predicted=0.314341, expected=0.314062
predicted=0.254107, expected=0.253829
predicted=0.192870, expected=0.192594
predicted=0.130872, expected=0.130599
predicted=0.068358, expected=0.068089
predicted=0.005574, expected=0.005310
predicted=-0.057232, expected=-0.057490
predicted=-0.119813, expected=-0.120064
predicted=-0.181920, expected=-0.182163
predicted=-0.243309, expected=-0.243544
predicted=-0.303739, expected=-0.303963
predicted=-0.362969, expected=-0.363183
predicted=-0.420767, expected=-0.420969
predicted=-0.476905, expected=-0.477094
predicted=-0.531160, expected=-0.531336
predicted=-0.583320, expected=-0.583481
predicted=-0.633177, expected=-0.633324
predicted=-0.680535, expected=-0.680667
predicted=-0.725208, expected=-0.725324
predicted=-0.767018, expected=-0.767118
predicted=-0.805802, expected=-0.805885
predicted=-0.841405, expected=-0.841471
predicted=-0.873688, expected=-0.873736
predicted=-0.902523, expected=-0.902554
predicted=-0.927796, expected=-0.927809
predicted=-0.949407, expected=-0.949402
predicted=-0.967272, expected=-0.967249
predicted=-0.981319, expected=-0.981278
predicted=-0.991494, expected=-0.991435
predicted=-0.997755, expected=-0.997679
predicted=-1.000079, expected=-0.999986
predicted=-0.998456, expected=-0.998346
predicted=-0.992893, expected=-0.992766
predicted=-0.983411, expected=-0.983268
predicted=-0.970048, expected=-0.969890
predicted=-0.952857, expected=-0.952684
predicted=-0.931905, expected=-0.931718
predicted=-0.907275, expected=-0.907075
predicted=-0.879065, expected=-0.878852
predicted=-0.847386, expected=-0.847161
predicted=-0.812363, expected=-0.812127
predicted=-0.774133, expected=-0.773887
predicted=-0.732849, expected=-0.732593
predicted=-0.688672, expected=-0.688408
predicted=-0.641777, expected=-0.641506
predicted=-0.592349, expected=-0.592073
predicted=-0.540584, expected=-0.540302
predicted=-0.486686, expected=-0.486400
predicted=-0.430866, expected=-0.430578
predicted=-0.373346, expected=-0.373056
predicted=-0.314353, expected=-0.314062
predicted=-0.254119, expected=-0.253829
predicted=-0.192883, expected=-0.192594
predicted=-0.130885, expected=-0.130599
predicted=-0.068370, expected=-0.068089
predicted=-0.005586, expected=-0.005310
predicted=0.057221, expected=0.057490
predicted=0.119801, expected=0.120064
predicted=0.181909, expected=0.182163
predicted=0.243299, expected=0.243544
predicted=0.303729, expected=0.303963
predicted=0.362960, expected=0.363183
predicted=0.420758, expected=0.420969
predicted=0.476896, expected=0.477094
predicted=0.531152, expected=0.531336
predicted=0.583312, expected=0.583481
predicted=0.633170, expected=0.633324
predicted=0.680529, expected=0.680667
predicted=0.725202, expected=0.725324
predicted=0.767014, expected=0.767118
predicted=0.805798, expected=0.805885
predicted=0.841402, expected=0.841471
predicted=0.873686, expected=0.873736
predicted=0.902521, expected=0.902554
predicted=0.927795, expected=0.927809
predicted=0.949407, expected=0.949402
predicted=0.967273, expected=0.967249
predicted=0.981321, expected=0.981278
predicted=0.991496, expected=0.991435
predicted=0.997758, expected=0.997679
predicted=1.000083, expected=0.999986
predicted=0.998461, expected=0.998346
predicted=0.992898, expected=0.992766
predicted=0.983417, expected=0.983268
predicted=0.970055, expected=0.969890
predicted=0.952864, expected=0.952684
predicted=0.931913, expected=0.931718
predicted=0.907284, expected=0.907075
predicted=0.879075, expected=0.878852
predicted=0.847396, expected=0.847161
predicted=0.812373, expected=0.812127
predicted=0.774144, expected=0.773887
predicted=0.732860, expected=0.732593
predicted=0.688683, expected=0.688408
predicted=0.641789, expected=0.641506
predicted=0.592362, expected=0.592073
predicted=0.540597, expected=0.540302
predicted=0.486698, expected=0.486400
predicted=0.430879, expected=0.430578
predicted=0.373359, expected=0.373056
predicted=0.314366, expected=0.314062
predicted=0.254132, expected=0.253829
predicted=0.192895, expected=0.192594
predicted=0.130897, expected=0.130599
predicted=0.068382, expected=0.068089
predicted=0.005598, expected=0.005310
predicted=-0.057209, expected=-0.057490
predicted=-0.119790, expected=-0.120064
predicted=-0.181898, expected=-0.182163
predicted=-0.243288, expected=-0.243544
predicted=-0.303718, expected=-0.303963
predicted=-0.362950, expected=-0.363183
predicted=-0.420749, expected=-0.420969
predicted=-0.476888, expected=-0.477094
predicted=-0.531144, expected=-0.531336
predicted=-0.583305, expected=-0.583481
predicted=-0.633163, expected=-0.633324
predicted=-0.680523, expected=-0.680667
predicted=-0.725197, expected=-0.725324
predicted=-0.767009, expected=-0.767118
predicted=-0.805794, expected=-0.805885
predicted=-0.841399, expected=-0.841471
predicted=-0.873684, expected=-0.873736
predicted=-0.902520, expected=-0.902554
predicted=-0.927795, expected=-0.927809
predicted=-0.949408, expected=-0.949402
predicted=-0.967274, expected=-0.967249
predicted=-0.981323, expected=-0.981278
predicted=-0.991499, expected=-0.991435
predicted=-0.997762, expected=-0.997679
predicted=-1.000087, expected=-0.999986
predicted=-0.998465, expected=-0.998346
predicted=-0.992904, expected=-0.992766
predicted=-0.983423, expected=-0.983268
predicted=-0.970062, expected=-0.969890
predicted=-0.952872, expected=-0.952684
predicted=-0.931921, expected=-0.931718
predicted=-0.907293, expected=-0.907075
predicted=-0.879084, expected=-0.878852
predicted=-0.847406, expected=-0.847161
predicted=-0.812383, expected=-0.812127
predicted=-0.774155, expected=-0.773887
predicted=-0.732871, expected=-0.732593
predicted=-0.688695, expected=-0.688408
predicted=-0.641801, expected=-0.641506
predicted=-0.592374, expected=-0.592073
predicted=-0.540609, expected=-0.540302
predicted=-0.486711, expected=-0.486400
predicted=-0.430891, expected=-0.430578
predicted=-0.373372, expected=-0.373056
predicted=-0.314379, expected=-0.314062
predicted=-0.254145, expected=-0.253829
predicted=-0.192908, expected=-0.192594
predicted=-0.130910, expected=-0.130599
predicted=-0.068395, expected=-0.068089
predicted=-0.005610, expected=-0.005310
predicted=0.057197, expected=0.057490
predicted=0.119778, expected=0.120064
predicted=0.181887, expected=0.182163
predicted=0.243278, expected=0.243544
predicted=0.303708, expected=0.303963
predicted=0.362940, expected=0.363183
predicted=0.420740, expected=0.420969
predicted=0.476879, expected=0.477094
predicted=0.531136, expected=0.531336
predicted=0.583298, expected=0.583481
predicted=0.633157, expected=0.633324
predicted=0.680517, expected=0.680667
predicted=0.725192, expected=0.725324
predicted=0.767005, expected=0.767118
predicted=0.805790, expected=0.805885
predicted=0.841396, expected=0.841471
Test RMSE: 0.000
In [121]:
plotly_actual_vs_predicted(index=ts1.index[last_points_for_testing:], actuals=test, predictions=predictions, title='Actuals vs Predictions', mode = 'lines')

It might look overfitted, but it is not. Because the very high correlation of this dataset, the AR model was able to capture the exact distribution on the data using 30 lags.

Analysing and forecasting the second dataset

Importing data

In [97]:
dataset2 = pd.read_csv('data2.csv', header=None, names=['valor'])

Building the time series

In [98]:
# Adiciona data no dataframe
dataset2['col_dt'] = datetime.datetime.now()
dataset2['linha'] = dataset2.index.astype(int)
dataset2['col_dt'] = dataset2.apply(lambda x: x.col_dt - timedelta(days = x.linha), axis=1)

# Grava a serie temporal no objeto ts1
ts2 = pd.Series(index=dataset2.col_dt, data=dataset2.valor.values)

# Verifica alguns registros
print(ts2.head())

# Sanity check para garantir que nossa serie nao misturou informacoes de datas diferentes
data = {'col_dt': ts2.index, 'valor': ts2.values}
ts2_df = pd.DataFrame(data=data)
sanity_df = pd.merge(dataset2, ts2_df, on=['col_dt'], how='inner')
print('Sanity test: {} inconsistencias encontradas'.format(len(sanity_df[sanity_df['valor_x'] != sanity_df['valor_y']])))
col_dt
2019-11-08 06:09:59.026699    1455.219971
2019-11-07 06:09:59.026699    1399.420044
2019-11-06 06:09:59.026699    1402.109985
2019-11-05 06:09:59.026699    1403.449951
2019-11-04 06:09:59.026699    1441.469971
dtype: float64
Sanity test: 0 inconsistencias encontradas
In [99]:
# Perform naise decomposition
result = seasonal_decompose(ts2, model='additive')
result.plot()
pyplot.show()

This serie presents some patterns on it's trend. There were a decreasing trend from the beginning of the serie to the half part. Then, we observe a quick increasing movement followed by a slowly decay. Finally another increasing pattern. The decreasing/increasing movements are not exactly "periodic enough" to associate with a season component.

From a business perspective, I tend to think this is related to non-periodic triggers (maybe creation of new lines of business, new products, etc...), which might change the behavior. However, in a global view, there is a decay movement.

Because the trend is NOT NULL, we might have a non-stationary serie. Lets see next.

Stationary test

In [100]:
stationary_test = adfuller(ts2, autolag='AIC')
stationary_test
Out[100]:
(0.014087354340239253,
 0.9596817103140847,
 18,
 4152,
 {'1%': -3.4319259507414803,
  '5%': -2.862236368514486,
  '10%': -2.567140683174717},
 34216.56169713592)

For each of those confidence intervals (99%, 95% and 90%), IF the critical value is greater than your test statistic (0.014), then the serie is stationary. In this case, this serie is NOT stationary with 99% of confidence

Autocorrelarion

This time, lets take a look in the autocorrelation plot using a pandas function.

In [102]:
autocorrelation_plot(ts2.values)
Out[102]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f18c58d43c8>

Since we have many point outside the dashed line, it looks there are a lot of correlation between lags. Lets again check the Pearson correlation between t and t+1

In [103]:
values = pd.DataFrame(ts2.values)
dataframe = pd.concat([values.shift(1), values], axis=1)
dataframe.columns = ['t', 't+1']
result = dataframe.corr()
result
Out[103]:
t t+1
t 1.000000 0.999013
t+1 0.999013 1.000000

Forecasting

This time we are going to use ARIMA, which is a generalization of the previous model we have used (AR), with some other componentes:

  • AR (AutoRegression) as I just said, same concept of the model we have built on the dataset 1. It used lag variables as features
  • I (Integrated) related to the use of differencing
  • MA (Moving Average) it used the dependency between observations and residual errors

Each of these components are hard set through params, used refered as ARIMA (p,d,q), where:

  • p is the number of lagged variables
  • d is the number of times that we apply differencing in the raw data
  • q is the size of the moving average window

Split train and test

In [ ]:
# split the data into train and test (in this case I am selecting the last 25% of the data for testing)
number_points = len(ts2.values)
last_points_for_testing = int(number_points * 0.25)

print ('-------Using {} rows for testing'.format(last_points_for_testing))

data_points = ts2.values
train, test = data_points[1:len(data_points)-last_points_for_testing], data_points[len(data_points)-last_points_for_testing:]

Model exploratory

Note: because this is a non-stationary TS, we have to apply differencing (will use order 1)

In [116]:
# fit model
model = ARIMA(train, order=(5,1,0))
model_fit = model.fit(disp=0)

# summary of fit model
print(model_fit.summary())

# line plot of residuals
residuals = pd.DataFrame(model_fit.resid)
residuals.plot()
pyplot.show()

# density plot of residuals
residuals.plot(kind='kde')
pyplot.show()

# summary stats of residuals
print(residuals.describe())
                             ARIMA Model Results                              
==============================================================================
Dep. Variable:                    D.y   No. Observations:                 3335
Model:                 ARIMA(5, 1, 0)   Log Likelihood              -13719.201
Method:                       css-mle   S.D. of innovations             14.802
Date:                Fri, 08 Nov 2019   AIC                          27452.402
Time:                        09:19:03   BIC                          27495.188
Sample:                             1   HQIC                         27467.709
                                                                              
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
const          0.0549      0.217      0.253      0.801      -0.371       0.481
ar.L1.D.y     -0.0857      0.017     -4.953      0.000      -0.120      -0.052
ar.L2.D.y     -0.0564      0.017     -3.249      0.001      -0.090      -0.022
ar.L3.D.y     -0.0010      0.017     -0.057      0.954      -0.035       0.033
ar.L4.D.y     -0.0007      0.017     -0.039      0.969      -0.035       0.033
ar.L5.D.y     -0.0356      0.017     -2.057      0.040      -0.070      -0.002
                                    Roots                                    
=============================================================================
                  Real          Imaginary           Modulus         Frequency
-----------------------------------------------------------------------------
AR.1            1.5671           -1.2841j            2.0260           -0.1093
AR.2            1.5671           +1.2841j            2.0260            0.1093
AR.3           -0.5925           -1.7670j            1.8637           -0.3015
AR.4           -0.5925           +1.7670j            1.8637            0.3015
AR.5           -1.9681           -0.0000j            1.9681           -0.5000
-----------------------------------------------------------------------------
                 0
count  3335.000000
mean      0.000573
std      14.804243
min    -106.974207
25%      -7.203683
50%       0.933140
75%       7.403102
max      97.353894

Quick residual analysis

Because the mean of my residuals is too close to zero and the distribution looks Gaussian, I dont thing there is any hard bias behind the scene. I will go ahead and perform my predictions.

However, usually we would spend more time on this step to find the best set of parameters for this model (probablt using grid-search)

Making predictions and evaluating

Now we will discuss a very important and particular aspect of TS modeling. Once we finish our modeling phase and we want to go to production predictions, we have to think about how the pipeline will work.

On regular ML models, we train the model just ONCE and then we expect to keep using the same model UNTIL signs of model drift appears. On TS, it is usually a good idea to keep retraining your model as soon as you receive more information. The reason is: as you go further with your predictions, your assertiveness tends to decrease.

Such issue did not happen on our first dataset just because it was following a very hard linear behavior. Then, even with very long predictions we had no damage on assertiveness. While here, on the second dataset, we can clearly see the decay of assertiveness.

  • Error in the first prediction was 1593 - 1585
  • Error in the last prediction was 2170 - 1629
  • RMSE: 359.885
In [118]:
# make predictions of the test period
predictions = model_fit.predict(start=len(train), end=len(train)+len(test)-1, dynamic=False, typ='levels')
for i in range(len(predictions)):
    print('predicted=%f, expected=%f' % (predictions[i], test[i]))

rmse = sqrt(mean_squared_error(test, predictions))
print('Test RMSE: %.3f' % rmse)
predicted=1585.613617, expected=1593.369995
predicted=1585.007585, expected=1588.849976
predicted=1584.871956, expected=1552.359985
predicted=1584.774338, expected=1574.569946
predicted=1584.175680, expected=1552.010010
predicted=1584.373214, expected=1541.609985
predicted=1584.476585, expected=1555.250000
predicted=1584.526811, expected=1562.500000
predicted=1584.585093, expected=1578.780029
predicted=1584.663100, expected=1578.790039
predicted=1584.710701, expected=1585.160034
predicted=1584.763179, expected=1582.239990
predicted=1584.818824, expected=1593.609985
predicted=1584.873652, expected=1597.569946
predicted=1584.927684, expected=1582.699951
predicted=1584.982908, expected=1597.589966
predicted=1585.037899, expected=1614.420044
predicted=1585.092731, expected=1617.500000
predicted=1585.147619, expected=1625.959961
predicted=1585.202538, expected=1632.689941
predicted=1585.257410, expected=1626.670044
predicted=1585.312292, expected=1633.699951
predicted=1585.367182, expected=1633.770020
predicted=1585.422068, expected=1650.339966
predicted=1585.476953, expected=1658.780029
predicted=1585.531841, expected=1650.469971
predicted=1585.586727, expected=1667.469971
predicted=1585.641614, expected=1666.290039
predicted=1585.696500, expected=1669.160034
predicted=1585.751387, expected=1655.349976
predicted=1585.806273, expected=1650.510010
predicted=1585.861160, expected=1649.599976
predicted=1585.916046, expected=1660.060059
predicted=1585.970933, expected=1648.359985
predicted=1586.025819, expected=1654.410034
predicted=1586.080706, expected=1630.739990
predicted=1586.135592, expected=1640.420044
predicted=1586.190479, expected=1631.380005
predicted=1586.245365, expected=1608.900024
predicted=1586.300252, expected=1622.560059
predicted=1586.355138, expected=1643.380005
predicted=1586.410025, expected=1642.810059
predicted=1586.464911, expected=1626.130005
predicted=1586.519798, expected=1612.520020
predicted=1586.574684, expected=1636.359985
predicted=1586.629571, expected=1626.729980
predicted=1586.684457, expected=1639.040039
predicted=1586.739344, expected=1651.810059
predicted=1586.794230, expected=1628.930054
predicted=1586.849117, expected=1588.189941
predicted=1586.904003, expected=1592.430054
predicted=1586.958890, expected=1573.089966
predicted=1587.013776, expected=1588.030029
predicted=1587.068663, expected=1603.260010
predicted=1587.123549, expected=1613.199951
predicted=1587.178436, expected=1606.280029
predicted=1587.233322, expected=1614.959961
predicted=1587.288209, expected=1614.079956
predicted=1587.343095, expected=1615.410034
predicted=1587.397982, expected=1631.890015
predicted=1587.452868, expected=1640.459961
predicted=1587.507755, expected=1652.319946
predicted=1587.562641, expected=1652.619995
predicted=1587.617528, expected=1675.020020
predicted=1587.672414, expected=1680.189941
predicted=1587.727301, expected=1682.500000
predicted=1587.782187, expected=1676.260010
predicted=1587.837074, expected=1680.910034
predicted=1587.891960, expected=1689.369995
predicted=1587.946847, expected=1692.089966
predicted=1588.001733, expected=1695.530029
predicted=1588.056620, expected=1692.390015
predicted=1588.111506, expected=1685.939941
predicted=1588.166393, expected=1690.250000
predicted=1588.221279, expected=1691.650024
predicted=1588.276166, expected=1685.329956
predicted=1588.331052, expected=1685.959961
predicted=1588.385939, expected=1685.729980
predicted=1588.440825, expected=1706.869995
predicted=1588.495712, expected=1709.670044
predicted=1588.550598, expected=1707.140015
predicted=1588.605485, expected=1697.369995
predicted=1588.660371, expected=1690.910034
predicted=1588.715258, expected=1697.479980
predicted=1588.770144, expected=1691.420044
predicted=1588.825031, expected=1689.469971
predicted=1588.879917, expected=1694.160034
predicted=1588.934804, expected=1685.390015
predicted=1588.989690, expected=1661.319946
predicted=1589.044577, expected=1655.829956
predicted=1589.099463, expected=1646.060059
predicted=1589.154350, expected=1652.349976
predicted=1589.209236, expected=1642.800049
predicted=1589.264123, expected=1656.959961
predicted=1589.319009, expected=1663.500000
predicted=1589.373896, expected=1656.780029
predicted=1589.428782, expected=1630.479980
predicted=1589.483669, expected=1634.959961
predicted=1589.538555, expected=1638.170044
predicted=1589.593442, expected=1632.969971
predicted=1589.648328, expected=1639.770020
predicted=1589.703215, expected=1653.079956
predicted=1589.758101, expected=1655.079956
predicted=1589.812988, expected=1655.170044
predicted=1589.867874, expected=1671.709961
predicted=1589.922761, expected=1683.989990
predicted=1589.977647, expected=1689.130005
predicted=1590.032534, expected=1683.420044
predicted=1590.087420, expected=1687.989990
predicted=1590.142307, expected=1697.599976
predicted=1590.197193, expected=1704.760010
predicted=1590.252080, expected=1725.520020
predicted=1590.306966, expected=1722.339966
predicted=1590.361853, expected=1709.910034
predicted=1590.416739, expected=1701.839966
predicted=1590.471626, expected=1697.420044
predicted=1590.526512, expected=1692.770020
predicted=1590.581399, expected=1698.670044
predicted=1590.636285, expected=1691.750000
predicted=1590.691172, expected=1681.550049
predicted=1590.746058, expected=1695.000000
predicted=1590.800945, expected=1693.869995
predicted=1590.855831, expected=1678.660034
predicted=1590.910718, expected=1690.500000
predicted=1590.965604, expected=1676.119995
predicted=1591.020491, expected=1655.449951
predicted=1591.075377, expected=1656.400024
predicted=1591.130264, expected=1692.560059
predicted=1591.185150, expected=1703.199951
predicted=1591.240037, expected=1710.140015
predicted=1591.294923, expected=1698.060059
predicted=1591.349810, expected=1721.540039
predicted=1591.404697, expected=1733.150024
predicted=1591.459583, expected=1744.500000
predicted=1591.514470, expected=1744.660034
predicted=1591.569356, expected=1754.670044
predicted=1591.624243, expected=1746.380005
predicted=1591.679129, expected=1752.069946
predicted=1591.734016, expected=1759.770020
predicted=1591.788902, expected=1762.109985
predicted=1591.843789, expected=1771.949951
predicted=1591.898675, expected=1763.310059
predicted=1591.953562, expected=1756.540039
predicted=1592.008448, expected=1761.640015
predicted=1592.063335, expected=1767.930054
predicted=1592.118221, expected=1762.969971
predicted=1592.173108, expected=1770.489990
predicted=1592.227994, expected=1747.150024
predicted=1592.282881, expected=1770.609985
predicted=1592.337767, expected=1771.890015
predicted=1592.392654, expected=1767.689941
predicted=1592.447540, expected=1782.000000
predicted=1592.502427, expected=1790.619995
predicted=1592.557313, expected=1798.180054
predicted=1592.612200, expected=1791.530029
predicted=1592.667086, expected=1787.869995
predicted=1592.721973, expected=1781.369995
predicted=1592.776859, expected=1795.849976
predicted=1592.831746, expected=1804.760010
predicted=1592.886632, expected=1802.479980
predicted=1592.941519, expected=1802.750000
predicted=1592.996405, expected=1807.229980
predicted=1593.051292, expected=1805.810059
predicted=1593.106178, expected=1800.900024
predicted=1593.161065, expected=1795.150024
predicted=1593.215951, expected=1792.810059
predicted=1593.270838, expected=1785.030029
predicted=1593.325724, expected=1805.089966
predicted=1593.380611, expected=1808.369995
predicted=1593.435497, expected=1802.619995
predicted=1593.490384, expected=1782.219971
predicted=1593.545270, expected=1775.500000
predicted=1593.600157, expected=1775.319946
predicted=1593.655043, expected=1786.540039
predicted=1593.709930, expected=1781.000000
predicted=1593.764816, expected=1810.650024
predicted=1593.819703, expected=1809.599976
predicted=1593.874589, expected=1818.319946
predicted=1593.929476, expected=1827.989990
predicted=1593.984362, expected=1833.319946
predicted=1594.039249, expected=1842.020020
predicted=1594.094135, expected=1841.400024
predicted=1594.149022, expected=1841.069946
predicted=1594.203908, expected=1848.359985
predicted=1594.258795, expected=1831.979980
predicted=1594.313681, expected=1831.369995
predicted=1594.368568, expected=1826.770020
predicted=1594.423454, expected=1837.880005
predicted=1594.478341, expected=1837.489990
predicted=1594.533227, expected=1838.130005
predicted=1594.588114, expected=1842.369995
predicted=1594.643000, expected=1819.199951
predicted=1594.697887, expected=1838.880005
predicted=1594.752773, expected=1848.380005
predicted=1594.807660, expected=1845.890015
predicted=1594.862546, expected=1838.699951
predicted=1594.917433, expected=1843.800049
predicted=1594.972319, expected=1844.859985
predicted=1595.027206, expected=1828.459961
predicted=1595.082092, expected=1790.290039
predicted=1595.136979, expected=1781.560059
predicted=1595.191865, expected=1792.500000
predicted=1595.246752, expected=1774.199951
predicted=1595.301638, expected=1794.189941
predicted=1595.356525, expected=1782.589966
predicted=1595.411411, expected=1741.890015
predicted=1595.466298, expected=1755.199951
predicted=1595.521184, expected=1751.640015
predicted=1595.576071, expected=1773.430054
predicted=1595.630957, expected=1797.020020
predicted=1595.685844, expected=1799.839966
predicted=1595.740730, expected=1819.750000
predicted=1595.795617, expected=1819.260010
predicted=1595.850503, expected=1829.829956
predicted=1595.905390, expected=1838.630005
predicted=1595.960276, expected=1840.760010
predicted=1596.015163, expected=1828.750000
predicted=1596.070049, expected=1839.780029
predicted=1596.124936, expected=1836.250000
predicted=1596.179822, expected=1847.609985
predicted=1596.234709, expected=1845.119995
predicted=1596.289595, expected=1845.160034
predicted=1596.344482, expected=1854.290039
predicted=1596.399368, expected=1859.449951
predicted=1596.454255, expected=1845.729980
predicted=1596.509141, expected=1873.910034
predicted=1596.564028, expected=1873.810059
predicted=1596.618914, expected=1877.030029
predicted=1596.673801, expected=1878.040039
predicted=1596.728687, expected=1877.170044
predicted=1596.783574, expected=1867.630005
predicted=1596.838460, expected=1868.199951
predicted=1596.893347, expected=1846.339966
predicted=1596.948233, expected=1841.130005
predicted=1597.003120, expected=1858.829956
predicted=1597.058006, expected=1872.250000
predicted=1597.112893, expected=1860.770020
predicted=1597.167779, expected=1872.010010
predicted=1597.222666, expected=1866.520020
predicted=1597.277552, expected=1857.439941
predicted=1597.332439, expected=1865.619995
predicted=1597.387325, expected=1852.560059
predicted=1597.442212, expected=1849.040039
predicted=1597.497098, expected=1857.619995
predicted=1597.551985, expected=1872.339966
predicted=1597.606871, expected=1885.520020
predicted=1597.661758, expected=1890.900024
predicted=1597.716644, expected=1888.770020
predicted=1597.771531, expected=1865.089966
predicted=1597.826417, expected=1845.040039
predicted=1597.881304, expected=1851.959961
predicted=1597.936190, expected=1872.180054
predicted=1597.991077, expected=1833.079956
predicted=1598.045963, expected=1815.689941
predicted=1598.100850, expected=1830.609985
predicted=1598.155736, expected=1842.979980
predicted=1598.210623, expected=1862.310059
predicted=1598.265509, expected=1864.849976
predicted=1598.320396, expected=1871.890015
predicted=1598.375282, expected=1879.550049
predicted=1598.430169, expected=1875.390015
predicted=1598.485055, expected=1878.609985
predicted=1598.539942, expected=1863.400024
predicted=1598.594828, expected=1869.430054
predicted=1598.649715, expected=1878.329956
predicted=1598.704601, expected=1883.949951
predicted=1598.759488, expected=1883.680054
predicted=1598.814374, expected=1881.140015
predicted=1598.869261, expected=1884.660034
predicted=1598.924147, expected=1867.719971
predicted=1598.979034, expected=1878.209961
predicted=1599.033920, expected=1875.630005
predicted=1599.088807, expected=1878.479980
predicted=1599.143694, expected=1896.650024
predicted=1599.198580, expected=1897.449951
predicted=1599.253467, expected=1888.530029
predicted=1599.308353, expected=1870.849976
predicted=1599.363240, expected=1877.859985
predicted=1599.418126, expected=1885.079956
predicted=1599.473013, expected=1872.829956
predicted=1599.527899, expected=1888.030029
predicted=1599.582786, expected=1892.489990
predicted=1599.637672, expected=1900.530029
predicted=1599.692559, expected=1911.910034
predicted=1599.747445, expected=1909.780029
predicted=1599.802332, expected=1920.030029
predicted=1599.857218, expected=1923.569946
predicted=1599.912105, expected=1924.969971
predicted=1599.966991, expected=1924.239990
predicted=1600.021878, expected=1927.880005
predicted=1600.076764, expected=1940.459961
predicted=1600.131651, expected=1949.439941
predicted=1600.186537, expected=1951.270020
predicted=1600.241424, expected=1950.790039
predicted=1600.296310, expected=1943.890015
predicted=1600.351197, expected=1930.109985
predicted=1600.406083, expected=1936.160034
predicted=1600.460970, expected=1937.780029
predicted=1600.515856, expected=1941.989990
predicted=1600.570743, expected=1956.979980
predicted=1600.625629, expected=1959.479980
predicted=1600.680516, expected=1962.869995
predicted=1600.735402, expected=1962.609985
predicted=1600.790289, expected=1949.979980
predicted=1600.845175, expected=1959.530029
predicted=1600.900062, expected=1957.219971
predicted=1600.954948, expected=1960.959961
predicted=1601.009835, expected=1960.229980
predicted=1601.064721, expected=1973.319946
predicted=1601.119608, expected=1974.619995
predicted=1601.174494, expected=1985.439941
predicted=1601.229381, expected=1977.650024
predicted=1601.284267, expected=1963.709961
predicted=1601.339154, expected=1972.829956
predicted=1601.394040, expected=1964.680054
predicted=1601.448927, expected=1967.569946
predicted=1601.503813, expected=1977.099976
predicted=1601.558700, expected=1973.280029
predicted=1601.613586, expected=1981.569946
predicted=1601.668473, expected=1958.119995
predicted=1601.723359, expected=1978.219971
predicted=1601.778246, expected=1973.630005
predicted=1601.833132, expected=1983.530029
predicted=1601.888019, expected=1987.010010
predicted=1601.942905, expected=1987.979980
predicted=1601.997792, expected=1978.339966
predicted=1602.052678, expected=1978.910034
predicted=1602.107565, expected=1969.949951
predicted=1602.162451, expected=1970.069946
predicted=1602.217338, expected=1930.670044
predicted=1602.272224, expected=1925.150024
predicted=1602.327111, expected=1938.989990
predicted=1602.381997, expected=1920.209961
predicted=1602.436884, expected=1920.239990
predicted=1602.491770, expected=1909.569946
predicted=1602.546657, expected=1931.589966
predicted=1602.601543, expected=1936.920044
predicted=1602.656430, expected=1933.750000
predicted=1602.711316, expected=1946.719971
predicted=1602.766203, expected=1955.180054
predicted=1602.821089, expected=1955.060059
predicted=1602.875976, expected=1971.739990
predicted=1602.930862, expected=1981.599976
predicted=1602.985749, expected=1986.510010
predicted=1603.040635, expected=1992.369995
predicted=1603.095522, expected=1988.400024
predicted=1603.150408, expected=1997.920044
predicted=1603.205295, expected=2000.020020
predicted=1603.260181, expected=2000.119995
predicted=1603.315068, expected=1996.739990
predicted=1603.369954, expected=2003.369995
predicted=1603.424841, expected=2002.280029
predicted=1603.479727, expected=2000.719971
predicted=1603.534614, expected=1997.650024
predicted=1603.589500, expected=2007.709961
predicted=1603.644387, expected=2001.540039
predicted=1603.699273, expected=1988.439941
predicted=1603.754160, expected=1995.689941
predicted=1603.809046, expected=1997.449951
predicted=1603.863933, expected=1985.540039
predicted=1603.918819, expected=1984.130005
predicted=1603.973706, expected=1998.979980
predicted=1604.028592, expected=2001.569946
predicted=1604.083479, expected=2011.359985
predicted=1604.138365, expected=2010.400024
predicted=1604.193252, expected=1994.290039
predicted=1604.248138, expected=1982.770020
predicted=1604.303025, expected=1998.300049
predicted=1604.357911, expected=1965.989990
predicted=1604.412798, expected=1982.849976
predicted=1604.467684, expected=1977.800049
predicted=1604.522571, expected=1972.290039
predicted=1604.577457, expected=1946.160034
predicted=1604.632344, expected=1946.170044
predicted=1604.687230, expected=1967.900024
predicted=1604.742117, expected=1964.819946
predicted=1604.797003, expected=1935.099976
predicted=1604.851890, expected=1968.890015
predicted=1604.906776, expected=1928.209961
predicted=1604.961663, expected=1906.130005
predicted=1605.016549, expected=1874.739990
predicted=1605.071436, expected=1877.699951
predicted=1605.126322, expected=1862.489990
predicted=1605.181209, expected=1862.760010
predicted=1605.236095, expected=1886.760010
predicted=1605.290982, expected=1904.010010
predicted=1605.345868, expected=1941.280029
predicted=1605.400755, expected=1927.109985
predicted=1605.455641, expected=1950.819946
predicted=1605.510528, expected=1964.579956
predicted=1605.565414, expected=1961.630005
predicted=1605.620301, expected=1985.050049
predicted=1605.675187, expected=1982.300049
predicted=1605.730074, expected=1994.650024
predicted=1605.784960, expected=2018.050049
predicted=1605.839847, expected=2017.810059
predicted=1605.894733, expected=2012.099976
predicted=1605.949620, expected=2023.569946
predicted=1606.004506, expected=2031.209961
predicted=1606.059393, expected=2031.920044
predicted=1606.114279, expected=2038.260010
predicted=1606.169166, expected=2039.680054
predicted=1606.224052, expected=2038.250000
predicted=1606.278939, expected=2039.329956
predicted=1606.333825, expected=2039.819946
predicted=1606.388712, expected=2041.319946
predicted=1606.443598, expected=2051.800049
predicted=1606.498485, expected=2048.719971
predicted=1606.553371, expected=2052.750000
predicted=1606.608258, expected=2063.500000
predicted=1606.663144, expected=2069.409912
predicted=1606.718031, expected=2067.030029
predicted=1606.772917, expected=2072.830078
predicted=1606.827804, expected=2067.560059
predicted=1606.882691, expected=2053.439941
predicted=1606.937577, expected=2066.550049
predicted=1606.992464, expected=2074.330078
predicted=1607.047350, expected=2071.919922
predicted=1607.102237, expected=2075.370117
predicted=1607.157123, expected=2060.310059
predicted=1607.212010, expected=2059.820068
predicted=1607.266896, expected=2026.140015
predicted=1607.321783, expected=2035.329956
predicted=1607.376669, expected=2002.329956
predicted=1607.431556, expected=1989.630005
predicted=1607.486442, expected=1972.739990
predicted=1607.541329, expected=2012.890015
predicted=1607.596215, expected=2061.229980
predicted=1607.651102, expected=2070.649902
predicted=1607.705988, expected=2078.540039
predicted=1607.760875, expected=2082.169922
predicted=1607.815761, expected=2081.879883
predicted=1607.870648, expected=2088.770020
predicted=1607.925534, expected=2090.570068
predicted=1607.980421, expected=2080.350098
predicted=1608.035307, expected=2058.899902
predicted=1608.090194, expected=2058.199951
predicted=1608.145080, expected=2020.579956
predicted=1608.199967, expected=2002.609985
predicted=1608.254853, expected=2025.900024
predicted=1608.309740, expected=2062.139893
predicted=1608.364626, expected=2044.810059
predicted=1608.419513, expected=2028.260010
predicted=1608.474399, expected=2023.030029
predicted=1608.529286, expected=2011.270020
predicted=1608.584172, expected=1992.670044
predicted=1608.639059, expected=2019.420044
predicted=1608.693945, expected=2022.550049
predicted=1608.748832, expected=2032.119995
predicted=1608.803718, expected=2063.149902
predicted=1608.858605, expected=2051.820068
predicted=1608.913491, expected=2057.090088
predicted=1608.968378, expected=2029.550049
predicted=1609.023264, expected=2002.160034
predicted=1609.078151, expected=2021.250000
predicted=1609.133037, expected=1994.989990
predicted=1609.187924, expected=2020.849976
predicted=1609.242810, expected=2050.030029
predicted=1609.297697, expected=2041.510010
predicted=1609.352583, expected=2062.520020
predicted=1609.407470, expected=2055.469971
predicted=1609.462356, expected=2046.739990
predicted=1609.517243, expected=2068.590088
predicted=1609.572129, expected=2068.530029
predicted=1609.627016, expected=2088.479980
predicted=1609.681902, expected=2096.989990
predicted=1609.736789, expected=2100.340088
predicted=1609.791675, expected=2099.679932
predicted=1609.846562, expected=2097.449951
predicted=1609.901448, expected=2110.300049
predicted=1609.956335, expected=2109.659912
predicted=1610.011221, expected=2115.479980
predicted=1610.066108, expected=2113.860107
predicted=1610.120994, expected=2110.739990
predicted=1610.175881, expected=2104.500000
predicted=1610.230767, expected=2117.389893
predicted=1610.285654, expected=2107.780029
predicted=1610.340540, expected=2098.530029
predicted=1610.395427, expected=2101.040039
predicted=1610.450313, expected=2071.260010
predicted=1610.505200, expected=2079.429932
predicted=1610.560086, expected=2044.160034
predicted=1610.614973, expected=2040.239990
predicted=1610.669859, expected=2065.949951
predicted=1610.724746, expected=2053.399902
predicted=1610.779632, expected=2081.189941
predicted=1610.834519, expected=2074.280029
predicted=1610.889405, expected=2099.500000
predicted=1610.944292, expected=2089.270020
predicted=1610.999178, expected=2108.100098
predicted=1611.054065, expected=2104.419922
predicted=1611.108951, expected=2091.500000
predicted=1611.163838, expected=2061.050049
predicted=1611.218724, expected=2056.149902
predicted=1611.273611, expected=2061.020020
predicted=1611.328497, expected=2086.239990
predicted=1611.383384, expected=2067.889893
predicted=1611.438270, expected=2059.689941
predicted=1611.493157, expected=2066.959961
predicted=1611.548043, expected=2080.620117
predicted=1611.602930, expected=2076.330078
predicted=1611.657816, expected=2081.899902
predicted=1611.712703, expected=2091.179932
predicted=1611.767589, expected=2102.060059
predicted=1611.822476, expected=2092.429932
predicted=1611.877362, expected=2095.840088
predicted=1611.932249, expected=2106.629883
predicted=1611.987135, expected=2104.989990
predicted=1612.042022, expected=2081.179932
predicted=1612.096908, expected=2100.399902
predicted=1612.151795, expected=2097.290039
predicted=1612.206681, expected=2107.959961
predicted=1612.261568, expected=2112.929932
predicted=1612.316454, expected=2117.689941
predicted=1612.371341, expected=2108.919922
predicted=1612.426227, expected=2114.760010
predicted=1612.481114, expected=2106.850098
predicted=1612.536000, expected=2085.510010
predicted=1612.590887, expected=2108.290039
predicted=1612.645773, expected=2114.489990
predicted=1612.700660, expected=2089.459961
predicted=1612.755546, expected=2080.149902
predicted=1612.810433, expected=2088.000000
predicted=1612.865319, expected=2116.100098
predicted=1612.920206, expected=2105.330078
predicted=1612.975092, expected=2099.120117
predicted=1613.029979, expected=2098.479980
predicted=1613.084865, expected=2121.100098
predicted=1613.139752, expected=2122.729980
predicted=1613.194638, expected=2129.199951
predicted=1613.249525, expected=2127.830078
predicted=1613.304411, expected=2125.850098
predicted=1613.359298, expected=2130.820068
predicted=1613.414184, expected=2126.060059
predicted=1613.469071, expected=2104.199951
predicted=1613.523957, expected=2123.479980
predicted=1613.578844, expected=2120.790039
predicted=1613.633730, expected=2107.389893
predicted=1613.688617, expected=2111.729980
predicted=1613.743503, expected=2109.600098
predicted=1613.798390, expected=2114.070068
predicted=1613.853276, expected=2095.840088
predicted=1613.908163, expected=2092.830078
predicted=1613.963049, expected=2079.280029
predicted=1614.017936, expected=2080.149902
predicted=1614.072822, expected=2105.199951
predicted=1614.127709, expected=2108.860107
predicted=1614.182595, expected=2094.110107
predicted=1614.237482, expected=2084.429932
predicted=1614.292368, expected=2096.290039
predicted=1614.347255, expected=2100.439941
predicted=1614.402141, expected=2121.239990
predicted=1614.457028, expected=2109.989990
predicted=1614.511914, expected=2122.850098
predicted=1614.566801, expected=2124.199951
predicted=1614.621688, expected=2108.580078
predicted=1614.676574, expected=2102.310059
predicted=1614.731461, expected=2101.489990
predicted=1614.786347, expected=2057.639893
predicted=1614.841234, expected=2063.110107
predicted=1614.896120, expected=2077.419922
predicted=1614.951007, expected=2076.780029
predicted=1615.005893, expected=2068.760010
predicted=1615.060780, expected=2081.340088
predicted=1615.115666, expected=2046.680054
predicted=1615.170553, expected=2051.310059
predicted=1615.225439, expected=2076.620117
predicted=1615.280326, expected=2099.600098
predicted=1615.335212, expected=2108.949951
predicted=1615.390099, expected=2107.399902
predicted=1615.444985, expected=2124.290039
predicted=1615.499872, expected=2126.639893
predicted=1615.554758, expected=2128.280029
predicted=1615.609645, expected=2119.209961
predicted=1615.664531, expected=2114.149902
predicted=1615.719418, expected=2102.149902
predicted=1615.774304, expected=2079.649902
predicted=1615.829191, expected=2067.639893
predicted=1615.884077, expected=2093.250000
predicted=1615.938964, expected=2108.570068
predicted=1615.993850, expected=2108.629883
predicted=1616.048737, expected=2103.840088
predicted=1616.103623, expected=2098.040039
predicted=1616.158510, expected=2093.320068
predicted=1616.213396, expected=2099.840088
predicted=1616.268283, expected=2083.560059
predicted=1616.323169, expected=2077.570068
predicted=1616.378056, expected=2104.179932
predicted=1616.432942, expected=2084.070068
predicted=1616.487829, expected=2086.050049
predicted=1616.542715, expected=2083.389893
predicted=1616.597602, expected=2091.540039
predicted=1616.652488, expected=2102.439941
predicted=1616.707375, expected=2096.919922
predicted=1616.762261, expected=2079.610107
predicted=1616.817148, expected=2035.729980
predicted=1616.872034, expected=1970.890015
predicted=1616.926921, expected=1893.209961
predicted=1616.981807, expected=1867.609985
predicted=1617.036694, expected=1940.510010
predicted=1617.091580, expected=1987.660034
predicted=1617.146467, expected=1988.869995
predicted=1617.201353, expected=1972.180054
predicted=1617.256240, expected=1913.849976
predicted=1617.311126, expected=1948.859985
predicted=1617.366013, expected=1951.130005
predicted=1617.420899, expected=1921.219971
predicted=1617.475786, expected=1969.410034
predicted=1617.530672, expected=1942.040039
predicted=1617.585559, expected=1952.290039
predicted=1617.640445, expected=1961.050049
predicted=1617.695332, expected=1953.030029
predicted=1617.750218, expected=1978.089966
predicted=1617.805105, expected=1995.310059
predicted=1617.859991, expected=1990.199951
predicted=1617.914878, expected=1958.030029
predicted=1617.969764, expected=1966.969971
predicted=1618.024651, expected=1942.739990
predicted=1618.079537, expected=1938.760010
predicted=1618.134424, expected=1932.239990
predicted=1618.189310, expected=1931.339966
predicted=1618.244197, expected=1881.770020
predicted=1618.299083, expected=1884.089966
predicted=1618.353970, expected=1920.030029
predicted=1618.408856, expected=1923.819946
predicted=1618.463743, expected=1951.359985
predicted=1618.518629, expected=1987.050049
predicted=1618.573516, expected=1979.920044
predicted=1618.628402, expected=1995.829956
predicted=1618.683289, expected=2013.430054
predicted=1618.738175, expected=2014.890015
predicted=1618.793062, expected=2017.459961
predicted=1618.847948, expected=2003.689941
predicted=1618.902835, expected=1994.239990
predicted=1618.957721, expected=2023.859985
predicted=1619.012608, expected=2033.109985
predicted=1619.067494, expected=2033.660034
predicted=1619.122381, expected=2030.770020
predicted=1619.177267, expected=2018.939941
predicted=1619.232154, expected=2052.510010
predicted=1619.287040, expected=2075.149902
predicted=1619.341927, expected=2071.179932
predicted=1619.396813, expected=2065.889893
predicted=1619.451700, expected=2090.350098
predicted=1619.506586, expected=2089.409912
predicted=1619.561473, expected=2079.360107
predicted=1619.616359, expected=2104.050049
predicted=1619.671246, expected=2109.790039
predicted=1619.726132, expected=2102.310059
predicted=1619.781019, expected=2099.929932
predicted=1619.835905, expected=2099.199951
predicted=1619.890792, expected=2078.580078
predicted=1619.945678, expected=2081.719971
predicted=1620.000565, expected=2075.000000
predicted=1620.055451, expected=2045.969971
predicted=1620.110338, expected=2023.040039
predicted=1620.165224, expected=2053.189941
predicted=1620.220111, expected=2050.439941
predicted=1620.274997, expected=2083.580078
predicted=1620.329884, expected=2081.239990
predicted=1620.384770, expected=2089.169922
predicted=1620.439657, expected=2086.590088
predicted=1620.494543, expected=2089.139893
predicted=1620.549430, expected=2088.870117
predicted=1620.604316, expected=2090.110107
predicted=1620.659203, expected=2080.409912
predicted=1620.714089, expected=2102.629883
predicted=1620.768976, expected=2079.510010
predicted=1620.823862, expected=2049.620117
predicted=1620.878749, expected=2091.689941
predicted=1620.933635, expected=2077.070068
predicted=1620.988522, expected=2063.590088
predicted=1621.043408, expected=2047.619995
predicted=1621.098295, expected=2052.229980
predicted=1621.153181, expected=2012.369995
predicted=1621.208068, expected=2021.939941
predicted=1621.262954, expected=2043.410034
predicted=1621.317841, expected=2073.070068
predicted=1621.372727, expected=2041.890015
predicted=1621.427614, expected=2005.550049
predicted=1621.482500, expected=2021.150024
predicted=1621.537387, expected=2038.969971
predicted=1621.592273, expected=2064.290039
predicted=1621.647160, expected=2060.989990
predicted=1621.702046, expected=2056.500000
predicted=1621.756933, expected=2078.360107
predicted=1621.811819, expected=2063.360107
predicted=1621.866706, expected=2043.939941
predicted=1621.921592, expected=2012.660034
predicted=1621.976479, expected=2016.709961
predicted=1622.031365, expected=1990.260010
predicted=1622.086252, expected=1943.089966
predicted=1622.141138, expected=1922.030029
predicted=1622.196025, expected=1923.670044
predicted=1622.250911, expected=1938.680054
predicted=1622.305798, expected=1890.280029
predicted=1622.360685, expected=1921.839966
predicted=1622.415571, expected=1880.329956
predicted=1622.470458, expected=1881.329956
predicted=1622.525344, expected=1859.329956
predicted=1622.580231, expected=1868.989990
predicted=1622.635117, expected=1906.900024
predicted=1622.690004, expected=1877.079956
predicted=1622.744890, expected=1903.630005
predicted=1622.799777, expected=1882.949951
predicted=1622.854663, expected=1893.359985
predicted=1622.909550, expected=1940.239990
predicted=1622.964436, expected=1939.380005
predicted=1623.019323, expected=1903.030029
predicted=1623.074209, expected=1912.530029
predicted=1623.129096, expected=1915.449951
predicted=1623.183982, expected=1880.050049
predicted=1623.238869, expected=1853.439941
predicted=1623.293755, expected=1852.209961
predicted=1623.348642, expected=1851.859985
predicted=1623.403528, expected=1829.079956
predicted=1623.458415, expected=1864.780029
predicted=1623.513301, expected=1895.579956
predicted=1623.568188, expected=1926.819946
predicted=1623.623074, expected=1917.829956
predicted=1623.677961, expected=1917.780029
predicted=1623.732847, expected=1945.500000
predicted=1623.787734, expected=1921.270020
predicted=1623.842620, expected=1929.800049
predicted=1623.897507, expected=1951.699951
predicted=1623.952393, expected=1948.050049
predicted=1624.007280, expected=1932.229980
predicted=1624.062166, expected=1978.349976
predicted=1624.117053, expected=1986.449951
predicted=1624.171939, expected=1993.400024
predicted=1624.226826, expected=1999.989990
predicted=1624.281712, expected=2001.760010
predicted=1624.336599, expected=1979.260010
predicted=1624.391485, expected=1989.260010
predicted=1624.446372, expected=1989.569946
predicted=1624.501258, expected=2022.189941
predicted=1624.556145, expected=2019.640015
predicted=1624.611031, expected=2015.930054
predicted=1624.665918, expected=2027.219971
predicted=1624.720804, expected=2040.589966
predicted=1624.775691, expected=2049.580078
predicted=1624.830577, expected=2051.600098
predicted=1624.885464, expected=2049.800049
predicted=1624.940350, expected=2036.709961
predicted=1624.995237, expected=2035.939941
predicted=1625.050123, expected=2037.050049
predicted=1625.105010, expected=2055.010010
predicted=1625.159896, expected=2063.949951
predicted=1625.214783, expected=2059.739990
predicted=1625.269669, expected=2072.780029
predicted=1625.324556, expected=2066.129883
predicted=1625.379442, expected=2045.170044
predicted=1625.434329, expected=2066.659912
predicted=1625.489215, expected=2041.910034
predicted=1625.544102, expected=2047.599976
predicted=1625.598988, expected=2041.989990
predicted=1625.653875, expected=2061.719971
predicted=1625.708761, expected=2082.419922
predicted=1625.763648, expected=2082.780029
predicted=1625.818534, expected=2080.729980
predicted=1625.873421, expected=2094.340088
predicted=1625.928307, expected=2100.800049
predicted=1625.983194, expected=2102.399902
predicted=1626.038080, expected=2091.479980
predicted=1626.092967, expected=2091.580078
predicted=1626.147853, expected=2087.790039
predicted=1626.202740, expected=2091.699951
predicted=1626.257626, expected=2095.149902
predicted=1626.312513, expected=2075.810059
predicted=1626.367399, expected=2065.300049
predicted=1626.422286, expected=2081.429932
predicted=1626.477172, expected=2063.370117
predicted=1626.532059, expected=2051.120117
predicted=1626.586945, expected=2050.629883
predicted=1626.641832, expected=2057.139893
predicted=1626.696718, expected=2058.689941
predicted=1626.751605, expected=2084.389893
predicted=1626.806491, expected=2064.459961
predicted=1626.861378, expected=2064.110107
predicted=1626.916264, expected=2046.609985
predicted=1626.971151, expected=2066.659912
predicted=1627.026037, expected=2047.209961
predicted=1627.080924, expected=2047.630005
predicted=1627.135810, expected=2040.040039
predicted=1627.190697, expected=2052.320068
predicted=1627.245583, expected=2048.040039
predicted=1627.300470, expected=2076.060059
predicted=1627.355356, expected=2090.540039
predicted=1627.410243, expected=2090.100098
predicted=1627.465129, expected=2099.060059
predicted=1627.520016, expected=2096.949951
predicted=1627.574902, expected=2099.330078
predicted=1627.629789, expected=2105.260010
predicted=1627.684675, expected=2099.129883
predicted=1627.739562, expected=2109.409912
predicted=1627.794448, expected=2112.129883
predicted=1627.849335, expected=2119.120117
predicted=1627.904221, expected=2115.479980
predicted=1627.959108, expected=2096.070068
predicted=1628.013994, expected=2079.060059
predicted=1628.068881, expected=2075.320068
predicted=1628.123767, expected=2071.500000
predicted=1628.178654, expected=2077.989990
predicted=1628.233540, expected=2071.219971
predicted=1628.288427, expected=2083.250000
predicted=1628.343313, expected=2088.899902
predicted=1628.398200, expected=2085.449951
predicted=1628.453086, expected=2113.320068
predicted=1628.507973, expected=2037.410034
predicted=1628.562859, expected=2000.540039
predicted=1628.617746, expected=2036.089966
predicted=1628.672632, expected=2070.770020
predicted=1628.727519, expected=2098.860107
predicted=1628.782405, expected=2102.949951
predicted=1628.837292, expected=2088.550049
predicted=1628.892178, expected=2099.729980
predicted=1628.947065, expected=2097.899902
predicted=1629.001951, expected=2129.899902
predicted=1629.056838, expected=2137.159912
predicted=1629.111724, expected=2152.139893
predicted=1629.166611, expected=2152.429932
predicted=1629.221497, expected=2163.750000
predicted=1629.276384, expected=2161.739990
predicted=1629.331270, expected=2166.889893
predicted=1629.386157, expected=2163.780029
predicted=1629.441043, expected=2173.020020
predicted=1629.495930, expected=2165.169922
predicted=1629.550816, expected=2175.030029
predicted=1629.605703, expected=2168.479980
predicted=1629.660589, expected=2169.179932
predicted=1629.715476, expected=2166.580078
predicted=1629.770362, expected=2170.060059
predicted=1629.825249, expected=2173.600098
predicted=1629.880135, expected=2170.840088
Test RMSE: 359.885

Retraining your model everytime you have new data

Based on those evidences, retraining your model everytime you receive more data is a better approach.

Quick note: based on some prior experience, I have received some requests from users to make predictions for more than one time step. We have to well-communicate users about the "assertiveness behavior of TS". If you communicate an average RMSE, users might have been informed that that metric tends to be better on first time steps and worse on latest time steps.

If you want to get away from this issue, you can do a daily training and walk-forward validation as follow:

In [149]:
history = [x for x in train]
predictions = list()
confidence_lower = list()
confidence_upper = list()

# walk-forward validation
for t in range(len(test)):
    model = ARIMA(history, order=(5,1,0))
    model_fit = model.fit(disp=0)
    #output = model_fit.forecast()
    output, stderr, conf = model_fit.forecast(alpha = 0.05)
    yhat = output[0]
    predictions.append(yhat)
    confidence_lower.append(conf[0][0])
    confidence_upper.append(conf[0][1])
    obs = test[t]
    history.append(obs)
    print('predicted=%f, expected=%f' % (yhat, obs))
    
# evaluate forecasts
rmse = sqrt(mean_squared_error(test, predictions))
print('Test RMSE: %.3f' % rmse)

# plot forecasts against actual outcomes
pyplot.plot(test)
pyplot.plot(predictions, color='red')
pyplot.show()
predicted=1585.613617, expected=1593.369995
predicted=1592.103820, expected=1588.849976
predicted=1588.614568, expected=1552.359985
predicted=1555.554241, expected=1574.569946
predicted=1574.109269, expected=1552.010010
predicted=1552.641435, expected=1541.609985
predicted=1543.995379, expected=1555.250000
predicted=1556.037065, expected=1562.500000
predicted=1560.372042, expected=1578.780029
predicted=1577.848822, expected=1578.790039
predicted=1578.282268, expected=1585.160034
predicted=1584.143702, expected=1582.239990
predicted=1581.907343, expected=1593.609985
predicted=1592.256052, expected=1597.569946
predicted=1596.654386, expected=1582.699951
predicted=1583.587117, expected=1597.589966
predicted=1597.288708, expected=1614.420044
predicted=1611.793490, expected=1617.500000
predicted=1616.221364, expected=1625.959961
predicted=1625.642377, expected=1632.689941
predicted=1631.155104, expected=1626.670044
predicted=1626.269137, expected=1633.699951
predicted=1633.387775, expected=1633.770020
predicted=1633.139691, expected=1650.339966
predicted=1648.754355, expected=1658.780029
predicted=1657.426212, expected=1650.469971
predicted=1650.539464, expected=1667.469971
predicted=1666.538895, expected=1666.290039
predicted=1664.935023, expected=1669.160034
predicted=1668.778666, expected=1655.349976
predicted=1656.752524, expected=1650.510010
predicted=1651.178736, expected=1649.599976
predicted=1650.080340, expected=1660.060059
predicted=1659.215003, expected=1648.359985
predicted=1649.372136, expected=1654.410034
predicted=1654.796578, expected=1630.739990
predicted=1632.569477, expected=1640.420044
predicted=1640.603019, expected=1631.380005
predicted=1632.153946, expected=1608.900024
predicted=1611.222052, expected=1622.560059
predicted=1623.524592, expected=1643.380005
predicted=1640.563637, expected=1642.810059
predicted=1642.116797, expected=1626.130005
predicted=1628.466859, expected=1612.520020
predicted=1614.213473, expected=1636.359985
predicted=1634.443231, expected=1626.729980
predicted=1626.329459, expected=1639.040039
predicted=1639.153385, expected=1651.810059
predicted=1650.564170, expected=1628.930054
predicted=1629.448268, expected=1588.189941
predicted=1593.405598, expected=1592.430054
predicted=1594.053943, expected=1573.089966
predicted=1574.170866, expected=1588.030029
predicted=1588.797446, expected=1603.260010
predicted=1602.697205, expected=1613.199951
predicted=1611.418004, expected=1606.280029
predicted=1607.042791, expected=1614.959961
predicted=1614.058603, expected=1614.079956
predicted=1613.151068, expected=1615.410034
predicted=1615.044080, expected=1631.890015
predicted=1630.715500, expected=1640.459961
predicted=1638.546615, expected=1652.319946
predicted=1650.911243, expected=1652.619995
predicted=1651.912734, expected=1675.020020
predicted=1672.533463, expected=1680.189941
predicted=1678.256024, expected=1682.500000
predicted=1681.647980, expected=1676.260010
predicted=1676.706654, expected=1680.910034
predicted=1680.112766, expected=1689.369995
predicted=1688.300801, expected=1692.089966
predicted=1691.404780, expected=1695.530029
predicted=1695.401387, expected=1692.390015
predicted=1692.380524, expected=1685.939941
predicted=1686.443318, expected=1690.250000
predicted=1690.240193, expected=1691.650024
predicted=1691.275952, expected=1685.329956
predicted=1686.010124, expected=1685.959961
predicted=1686.587878, expected=1685.729980
predicted=1685.658914, expected=1706.869995
predicted=1705.140037, expected=1709.670044
predicted=1708.590243, expected=1707.140015
predicted=1707.254791, expected=1697.369995
predicted=1698.419494, expected=1690.910034
predicted=1691.327114, expected=1697.479980
predicted=1697.301724, expected=1691.420044
predicted=1691.788449, expected=1689.469971
predicted=1690.433027, expected=1694.160034
predicted=1694.206004, expected=1685.390015
predicted=1685.746298, expected=1661.319946
predicted=1664.167315, expected=1655.829956
predicted=1657.806109, expected=1646.060059
predicted=1647.146439, expected=1652.349976
predicted=1652.815094, expected=1642.800049
predicted=1644.243215, expected=1656.959961
predicted=1656.581548, expected=1663.500000
predicted=1662.616703, expected=1656.780029
predicted=1656.837567, expected=1630.479980
predicted=1633.485535, expected=1634.959961
predicted=1635.606357, expected=1638.170044
predicted=1637.543860, expected=1632.969971
predicted=1633.583597, expected=1639.770020
predicted=1640.534259, expected=1653.079956
predicted=1651.494731, expected=1655.079956
predicted=1654.128111, expected=1655.170044
predicted=1655.302249, expected=1671.709961
predicted=1670.124118, expected=1683.989990
predicted=1681.627125, expected=1689.130005
predicted=1688.017558, expected=1683.420044
predicted=1683.676220, expected=1687.989990
predicted=1687.385443, expected=1697.599976
predicted=1696.184944, expected=1704.760010
predicted=1703.534030, expected=1725.520020
predicted=1723.692689, expected=1722.339966
predicted=1721.383535, expected=1709.910034
predicted=1710.846783, expected=1701.839966
predicted=1703.028952, expected=1697.420044
predicted=1697.588684, expected=1692.770020
predicted=1693.647296, expected=1698.670044
predicted=1699.017571, expected=1691.750000
predicted=1692.412399, expected=1681.550049
predicted=1683.043561, expected=1695.000000
predicted=1694.717822, expected=1693.869995
predicted=1693.116233, expected=1678.660034
predicted=1680.344633, expected=1690.500000
predicted=1690.823747, expected=1676.119995
predicted=1676.278794, expected=1655.449951
predicted=1658.128173, expected=1656.400024
predicted=1658.126278, expected=1692.560059
predicted=1689.163256, expected=1703.199951
predicted=1700.932634, expected=1710.140015
predicted=1709.801194, expected=1698.060059
predicted=1698.659023, expected=1721.540039
predicted=1718.988321, expected=1733.150024
predicted=1730.594727, expected=1744.500000
predicted=1742.728630, expected=1744.660034
predicted=1744.554894, expected=1754.670044
predicted=1753.071464, expected=1746.380005
predicted=1746.193675, expected=1752.069946
predicted=1751.743051, expected=1759.770020
predicted=1758.945654, expected=1762.109985
predicted=1761.225779, expected=1771.949951
predicted=1771.427976, expected=1763.310059
predicted=1763.387931, expected=1756.540039
predicted=1757.405158, expected=1761.640015
predicted=1761.646869, expected=1767.930054
predicted=1766.893864, expected=1762.969971
predicted=1763.459245, expected=1770.489990
predicted=1770.511711, expected=1747.150024
predicted=1748.606916, expected=1770.609985
predicted=1769.825627, expected=1771.890015
predicted=1770.824918, expected=1767.689941
predicted=1767.773519, expected=1782.000000
predicted=1782.009721, expected=1790.619995
predicted=1788.377033, expected=1798.180054
predicted=1797.137948, expected=1791.530029
predicted=1791.938956, expected=1787.869995
predicted=1788.146419, expected=1781.369995
predicted=1781.939989, expected=1795.849976
predicted=1794.865792, expected=1804.760010
predicted=1803.595341, expected=1802.479980
predicted=1802.415427, expected=1802.750000
predicted=1803.211514, expected=1807.229980
predicted=1806.457425, expected=1805.810059
predicted=1805.490319, expected=1800.900024
predicted=1801.599229, expected=1795.150024
predicted=1796.029670, expected=1792.810059
predicted=1793.303492, expected=1785.030029
predicted=1785.998090, expected=1805.089966
predicted=1804.168366, expected=1808.369995
predicted=1807.334147, expected=1802.619995
predicted=1803.101620, expected=1782.219971
predicted=1784.648895, expected=1775.500000
predicted=1776.614045, expected=1775.319946
predicted=1775.765812, expected=1786.540039
predicted=1785.978238, expected=1781.000000
predicted=1781.697980, expected=1810.650024
predicted=1808.845579, expected=1809.599976
predicted=1808.186193, expected=1818.319946
predicted=1817.299818, expected=1827.989990
predicted=1827.044776, expected=1833.319946
predicted=1831.374160, expected=1842.020020
predicted=1841.170793, expected=1841.400024
predicted=1840.790441, expected=1841.069946
predicted=1840.908692, expected=1848.359985
predicted=1847.730934, expected=1831.979980
predicted=1832.766748, expected=1831.369995
predicted=1832.476769, expected=1826.770020
predicted=1827.371560, expected=1837.880005
predicted=1837.090555, expected=1837.489990
predicted=1837.664531, expected=1838.130005
predicted=1838.242195, expected=1842.369995
predicted=1842.297944, expected=1819.199951
predicted=1820.629612, expected=1838.880005
predicted=1838.656842, expected=1848.380005
predicted=1846.670238, expected=1845.890015
predicted=1845.534745, expected=1838.699951
predicted=1840.398689, expected=1843.800049
predicted=1843.210569, expected=1844.859985
predicted=1844.308291, expected=1828.459961
predicted=1829.995522, expected=1790.290039
predicted=1794.750860, expected=1781.560059
predicted=1784.362110, expected=1792.500000
predicted=1792.265710, expected=1774.199951
predicted=1775.802244, expected=1794.189941
predicted=1795.050299, expected=1782.589966
predicted=1782.932212, expected=1741.890015
predicted=1745.615369, expected=1755.199951
predicted=1757.133120, expected=1751.640015
predicted=1750.645465, expected=1773.430054
predicted=1772.363338, expected=1797.020020
predicted=1795.392213, expected=1799.839966
predicted=1797.935130, expected=1819.750000
predicted=1818.145479, expected=1819.260010
predicted=1817.527220, expected=1829.829956
predicted=1828.293650, expected=1838.630005
predicted=1837.342856, expected=1840.760010
predicted=1839.567954, expected=1828.750000
predicted=1829.753116, expected=1839.780029
predicted=1839.290517, expected=1836.250000
predicted=1835.782460, expected=1847.609985
predicted=1846.953904, expected=1845.119995
predicted=1845.252689, expected=1845.160034
predicted=1845.057061, expected=1854.290039
predicted=1853.788153, expected=1859.449951
predicted=1858.295774, expected=1845.729980
predicted=1846.806897, expected=1873.910034
predicted=1872.457977, expected=1873.810059
predicted=1872.125519, expected=1877.030029
predicted=1876.759953, expected=1878.040039
predicted=1878.368615, expected=1877.170044
predicted=1876.371983, expected=1867.630005
predicted=1868.618121, expected=1868.199951
predicted=1868.708947, expected=1846.339966
predicted=1848.233516, expected=1841.130005
predicted=1842.925934, expected=1858.829956
predicted=1858.136715, expected=1872.250000
predicted=1870.362228, expected=1860.770020
predicted=1861.902573, expected=1872.010010
predicted=1872.009025, expected=1866.520020
predicted=1865.871632, expected=1857.439941
predicted=1858.181466, expected=1865.619995
predicted=1865.974950, expected=1852.560059
predicted=1852.964547, expected=1849.040039
predicted=1850.402005, expected=1857.619995
predicted=1857.553398, expected=1872.339966
predicted=1870.547348, expected=1885.520020
predicted=1884.258889, expected=1890.900024
predicted=1890.002022, expected=1888.770020
predicted=1888.467861, expected=1865.089966
predicted=1866.736588, expected=1845.040039
predicted=1847.616921, expected=1851.959961
predicted=1852.471367, expected=1872.180054
predicted=1870.494327, expected=1833.079956
predicted=1836.222502, expected=1815.689941
predicted=1820.107416, expected=1830.609985
predicted=1830.248758, expected=1842.979980
predicted=1840.627957, expected=1862.310059
predicted=1861.574066, expected=1864.849976
predicted=1864.259975, expected=1871.890015
predicted=1870.747647, expected=1879.550049
predicted=1878.211876, expected=1875.390015
predicted=1874.761796, expected=1878.609985
predicted=1878.623934, expected=1863.400024
predicted=1864.347163, expected=1869.430054
predicted=1869.680545, expected=1878.329956
predicted=1877.581435, expected=1883.949951
predicted=1883.067976, expected=1883.680054
predicted=1884.048823, expected=1881.140015
predicted=1881.277968, expected=1884.660034
predicted=1884.351008, expected=1867.719971
predicted=1868.859751, expected=1878.209961
predicted=1878.469540, expected=1875.630005
predicted=1875.513710, expected=1878.479980
predicted=1878.450389, expected=1896.650024
predicted=1895.736630, expected=1897.449951
predicted=1896.166971, expected=1888.530029
predicted=1889.427857, expected=1870.849976
predicted=1872.785798, expected=1877.859985
predicted=1877.816433, expected=1885.079956
predicted=1884.285931, expected=1872.829956
predicted=1873.920226, expected=1888.030029
predicted=1888.224739, expected=1892.489990
predicted=1891.179474, expected=1900.530029
predicted=1899.544099, expected=1911.910034
predicted=1911.073853, expected=1909.780029
predicted=1908.924265, expected=1920.030029
predicted=1919.290770, expected=1923.569946
predicted=1922.568448, expected=1924.969971
predicted=1924.428288, expected=1924.239990
predicted=1924.430725, expected=1927.880005
predicted=1927.429169, expected=1940.459961
predicted=1939.283744, expected=1949.439941
predicted=1948.135848, expected=1951.270020
predicted=1950.799701, expected=1950.790039
predicted=1950.735187, expected=1943.890015
predicted=1944.190880, expected=1930.109985
predicted=1931.465741, expected=1936.160034
predicted=1936.560193, expected=1937.780029
predicted=1937.539005, expected=1941.989990
predicted=1942.006748, expected=1956.979980
predicted=1956.168455, expected=1959.479980
predicted=1958.399791, expected=1962.869995
predicted=1962.549770, expected=1962.609985
predicted=1962.424245, expected=1949.979980
predicted=1950.656742, expected=1959.530029
predicted=1959.543972, expected=1957.219971
predicted=1956.952489, expected=1960.959961
predicted=1961.007935, expected=1960.229980
predicted=1960.668843, expected=1973.319946
predicted=1972.154003, expected=1974.619995
predicted=1974.039506, expected=1985.439941
predicted=1984.534932, expected=1977.650024
predicted=1977.846605, expected=1963.709961
predicted=1964.975655, expected=1972.829956
predicted=1972.979913, expected=1964.680054
predicted=1964.685145, expected=1967.569946
predicted=1968.281503, expected=1977.099976
predicted=1976.812704, expected=1973.280029
predicted=1972.953639, expected=1981.569946
predicted=1981.556233, expected=1958.119995
predicted=1959.616930, expected=1978.219971
predicted=1977.729922, expected=1973.630005
predicted=1973.217659, expected=1983.530029
predicted=1982.918522, expected=1987.010010
predicted=1987.108148, expected=1987.979980
predicted=1987.202827, expected=1978.339966
predicted=1979.379743, expected=1978.910034
predicted=1979.221364, expected=1969.949951
predicted=1970.731201, expected=1970.069946
predicted=1970.737711, expected=1930.670044
predicted=1934.419417, expected=1925.150024
predicted=1927.923932, expected=1938.989990
predicted=1938.716895, expected=1920.209961
predicted=1921.288791, expected=1920.239990
predicted=1922.799774, expected=1909.569946
predicted=1910.767860, expected=1931.589966
predicted=1930.113024, expected=1936.920044
predicted=1936.101415, expected=1933.750000
predicted=1933.887063, expected=1946.719971
predicted=1946.288504, expected=1955.180054
predicted=1953.195777, expected=1955.060059
predicted=1954.599761, expected=1971.739990
predicted=1970.624962, expected=1981.599976
predicted=1979.611593, expected=1986.510010
predicted=1985.467302, expected=1992.369995
predicted=1991.771582, expected=1988.400024
predicted=1988.005416, expected=1997.920044
predicted=1997.204369, expected=2000.020020
predicted=1999.347589, expected=2000.119995
predicted=2000.000187, expected=1996.739990
predicted=1997.307868, expected=2003.369995
predicted=2002.880615, expected=2002.280029
predicted=2002.132464, expected=2000.719971
predicted=2001.099175, expected=1997.650024
predicted=1998.271942, expected=2007.709961
predicted=2007.028956, expected=2001.540039
predicted=2001.732052, expected=1988.439941
predicted=1990.086227, expected=1995.689941
predicted=1996.078824, expected=1997.449951
predicted=1996.783273, expected=1985.540039
predicted=1986.838073, expected=1984.130005
predicted=1985.501557, expected=1998.979980
predicted=1997.785966, expected=2001.569946
predicted=2000.702921, expected=2011.359985
predicted=2011.014789, expected=2010.400024
predicted=2010.152964, expected=1994.290039
predicted=1995.335941, expected=1982.770020
predicted=1984.659114, expected=1998.300049
predicted=1997.533115, expected=1965.989990
predicted=1968.046563, expected=1982.849976
predicted=1983.972407, expected=1977.800049
predicted=1977.851540, expected=1972.290039
predicted=1972.756419, expected=1946.160034
predicted=1949.880463, expected=1946.170044
predicted=1947.179613, expected=1967.900024
predicted=1966.481639, expected=1964.819946
predicted=1964.301062, expected=1935.099976
predicted=1938.761689, expected=1968.890015
predicted=1967.870139, expected=1928.209961
predicted=1929.259131, expected=1906.130005
predicted=1910.581221, expected=1874.739990
predicted=1879.697375, expected=1877.699951
predicted=1878.104503, expected=1862.489990
predicted=1865.164986, expected=1862.760010
predicted=1864.450584, expected=1886.760010
predicted=1886.033890, expected=1904.010010
predicted=1901.382434, expected=1941.280029
predicted=1938.096654, expected=1927.109985
predicted=1926.464548, expected=1950.819946
predicted=1948.898751, expected=1964.579956
predicted=1961.784438, expected=1961.630005
predicted=1960.041937, expected=1985.050049
predicted=1983.981883, expected=1982.300049
predicted=1980.691746, expected=1994.650024
predicted=1993.541081, expected=2018.050049
predicted=2015.809307, expected=2017.810059
predicted=2016.097403, expected=2012.099976
predicted=2012.928789, expected=2023.569946
predicted=2022.696624, expected=2031.209961
predicted=2029.398682, expected=2031.920044
predicted=2031.719816, expected=2038.260010
predicted=2038.130172, expected=2039.680054
predicted=2039.064032, expected=2038.250000
predicted=2038.256401, expected=2039.329956
predicted=2039.495074, expected=2039.819946
predicted=2039.705803, expected=2041.319946
predicted=2041.327358, expected=2051.800049
predicted=2051.125533, expected=2048.719971
predicted=2048.617323, expected=2052.750000
predicted=2052.797410, expected=2063.500000
predicted=2062.573266, expected=2069.409912
predicted=2068.260108, expected=2067.030029
predicted=2067.278974, expected=2072.830078
predicted=2072.576330, expected=2067.560059
predicted=2067.536521, expected=2053.439941
predicted=2054.869856, expected=2066.550049
predicted=2066.458474, expected=2074.330078
predicted=2072.999826, expected=2071.919922
predicted=2072.154478, expected=2075.370117
predicted=2075.931751, expected=2060.310059
predicted=2061.116219, expected=2059.820068
predicted=2060.567154, expected=2026.140015
predicted=2029.125839, expected=2035.329956
predicted=2036.320659, expected=2002.329956
predicted=2005.172327, expected=1989.630005
predicted=1992.449864, expected=1972.739990
predicted=1975.900922, expected=2012.890015
predicted=2010.188904, expected=2061.229980
predicted=2056.770257, expected=2070.649902
predicted=2068.293472, expected=2078.540039
predicted=2078.346563, expected=2082.169922
predicted=2080.276602, expected=2081.879883
predicted=2080.221658, expected=2088.770020
predicted=2088.130995, expected=2090.570068
predicted=2090.034906, expected=2080.350098
predicted=2081.158443, expected=2058.899902
predicted=2061.270941, expected=2058.199951
predicted=2059.211941, expected=2020.579956
predicted=2023.606658, expected=2002.609985
predicted=2006.276364, expected=2025.900024
predicted=2025.806389, expected=2062.139893
predicted=2058.494915, expected=2044.810059
predicted=2046.011438, expected=2028.260010
predicted=2031.267213, expected=2023.030029
predicted=2023.581734, expected=2011.270020
predicted=2011.286287, expected=1992.670044
predicted=1995.479675, expected=2019.420044
predicted=2019.094249, expected=2022.550049
predicted=2021.373567, expected=2032.119995
predicted=2031.914371, expected=2063.149902
predicted=2061.204174, expected=2051.820068
predicted=2050.355207, expected=2057.090088
predicted=2057.357235, expected=2029.550049
predicted=2031.153377, expected=2002.160034
predicted=2004.598892, expected=2021.250000
predicted=2021.712375, expected=1994.989990
predicted=1996.114384, expected=2020.849976
predicted=2021.468605, expected=2050.030029
predicted=2047.640595, expected=2041.510010
predicted=2040.402679, expected=2062.520020
predicted=2062.419036, expected=2055.469971
predicted=2054.025441, expected=2046.739990
predicted=2046.953542, expected=2068.590088
predicted=2067.677756, expected=2068.530029
predicted=2066.927191, expected=2088.479980
predicted=2087.529086, expected=2096.989990
predicted=2095.757919, expected=2100.340088
predicted=2099.148436, expected=2099.679932
predicted=2099.672947, expected=2097.449951
predicted=2097.084865, expected=2110.300049
predicted=2109.299328, expected=2109.659912
predicted=2109.189278, expected=2115.479980
predicted=2115.358475, expected=2113.860107
predicted=2113.916481, expected=2110.739990
predicted=2110.829822, expected=2104.500000
predicted=2105.322411, expected=2117.389893
predicted=2116.702497, expected=2107.780029
predicted=2108.186842, expected=2098.530029
predicted=2100.110238, expected=2101.040039
predicted=2101.612595, expected=2071.260010
predicted=2073.215487, expected=2079.429932
predicted=2080.839273, expected=2044.160034
predicted=2046.932783, expected=2040.239990
predicted=2042.505533, expected=2065.949951
predicted=2065.210807, expected=2053.399902
predicted=2053.333306, expected=2081.189941
predicted=2081.174471, expected=2074.280029
predicted=2073.604203, expected=2099.500000
predicted=2097.346399, expected=2089.270020
predicted=2089.340654, expected=2108.100098
predicted=2106.432752, expected=2104.419922
predicted=2104.111637, expected=2091.500000
predicted=2092.147408, expected=2061.050049
predicted=2064.448227, expected=2056.149902
predicted=2057.410124, expected=2061.020020
predicted=2061.213155, expected=2086.239990
predicted=2084.960580, expected=2067.889893
predicted=2069.535477, expected=2059.689941
predicted=2061.548901, expected=2066.959961
predicted=2066.575101, expected=2080.620117
predicted=2078.640853, expected=2076.330078
predicted=2076.965680, expected=2081.899902
predicted=2082.127786, expected=2091.179932
predicted=2090.045650, expected=2102.060059
predicted=2100.547216, expected=2092.429932
predicted=2093.029720, expected=2095.840088
predicted=2095.980657, expected=2106.629883
predicted=2105.422272, expected=2104.989990
predicted=2104.517944, expected=2081.179932
predicted=2083.674163, expected=2100.399902
predicted=2100.000006, expected=2097.290039
predicted=2096.437095, expected=2107.959961
predicted=2107.709127, expected=2112.929932
predicted=2112.994039, expected=2117.689941
predicted=2116.645437, expected=2108.919922
predicted=2109.663312, expected=2114.760010
predicted=2114.504945, expected=2106.850098
predicted=2107.198455, expected=2085.510010
predicted=2087.656129, expected=2108.290039
predicted=2107.944618, expected=2114.489990
predicted=2112.960764, expected=2089.459961
predicted=2091.845010, expected=2080.149902
predicted=2082.922678, expected=2088.000000
predicted=2087.100875, expected=2116.100098
predicted=2113.631107, expected=2105.330078
predicted=2106.021226, expected=2099.120117
predicted=2100.663288, expected=2098.479980
predicted=2098.593847, expected=2121.100098
predicted=2118.603178, expected=2122.729980
predicted=2122.143426, expected=2129.199951
predicted=2129.073195, expected=2127.830078
predicted=2127.765844, expected=2125.850098
predicted=2125.488327, expected=2130.820068
predicted=2130.651549, expected=2126.060059
predicted=2126.194985, expected=2104.199951
predicted=2106.431159, expected=2123.479980
predicted=2123.254531, expected=2120.790039
predicted=2120.120771, expected=2107.389893
predicted=2109.097353, expected=2111.729980
predicted=2112.923863, expected=2109.600098
predicted=2109.084936, expected=2114.070068
predicted=2114.197691, expected=2095.840088
predicted=2097.736725, expected=2092.830078
predicted=2094.017661, expected=2079.280029
predicted=2080.736708, expected=2080.149902
predicted=2080.853866, expected=2105.199951
predicted=2104.018300, expected=2108.860107
predicted=2107.751945, expected=2094.110107
predicted=2095.810678, expected=2084.429932
predicted=2085.968061, expected=2096.290039
predicted=2095.125530, expected=2100.439941
predicted=2099.675470, expected=2121.239990
predicted=2120.172183, expected=2109.989990
predicted=2110.362000, expected=2122.850098
predicted=2122.162224, expected=2124.199951
predicted=2123.440590, expected=2108.580078
predicted=2109.311600, expected=2102.310059
predicted=2104.108284, expected=2101.489990
predicted=2101.607353, expected=2057.639893
predicted=2061.380585, expected=2063.110107
predicted=2065.573298, expected=2077.419922
predicted=2076.396235, expected=2076.780029
predicted=2076.600568, expected=2068.760010
predicted=2071.185540, expected=2081.340088
predicted=2080.667646, expected=2046.680054
predicted=2048.505767, expected=2051.310059
predicted=2052.901399, expected=2076.620117
predicted=2074.737251, expected=2099.600098
predicted=2096.459227, expected=2108.949951
predicted=2108.552137, expected=2107.399902
predicted=2106.969478, expected=2124.290039
predicted=2122.214709, expected=2126.639893
predicted=2124.962555, expected=2128.280029
predicted=2127.942370, expected=2119.209961
predicted=2120.032741, expected=2114.149902
predicted=2114.581591, expected=2102.149902
predicted=2103.448250, expected=2079.649902
predicted=2082.180624, expected=2067.639893
predicted=2070.202532, expected=2093.250000
predicted=2092.266223, expected=2108.570068
predicted=2106.892758, expected=2108.629883
predicted=2108.996651, expected=2103.840088
predicted=2104.724177, expected=2098.040039
predicted=2097.903493, expected=2093.320068
predicted=2093.612143, expected=2099.840088
predicted=2099.798683, expected=2083.560059
predicted=2084.921919, expected=2077.570068
predicted=2079.281762, expected=2104.179932
predicted=2102.739106, expected=2084.070068
predicted=2084.388127, expected=2086.050049
predicted=2087.750739, expected=2083.389893
predicted=2083.752896, expected=2091.540039
predicted=2090.372868, expected=2102.439941
predicted=2102.108856, expected=2096.919922
predicted=2096.969409, expected=2079.610107
predicted=2081.498203, expected=2035.729980
predicted=2039.816387, expected=1970.890015
predicted=1977.575758, expected=1893.209961
predicted=1901.526177, expected=1867.609985
predicted=1873.024427, expected=1940.510010
predicted=1939.464022, expected=1987.660034
predicted=1985.358708, expected=1988.869995
predicted=1989.973183, expected=1972.180054
predicted=1973.032860, expected=1913.849976
predicted=1914.609702, expected=1948.859985
predicted=1948.124426, expected=1951.130005
predicted=1950.250100, expected=1921.219971
predicted=1924.424783, expected=1969.410034
predicted=1969.464584, expected=1942.040039
predicted=1940.515263, expected=1952.290039
predicted=1953.288572, expected=1961.050049
predicted=1960.385846, expected=1953.030029
predicted=1952.101222, expected=1978.089966
predicted=1977.621704, expected=1995.310059
predicted=1992.600449, expected=1990.199951
predicted=1989.563373, expected=1958.030029
predicted=1960.217168, expected=1966.969971
predicted=1966.981823, expected=1942.739990
predicted=1943.811780, expected=1938.760010
predicted=1941.204518, expected=1932.239990
predicted=1934.150341, expected=1931.339966
predicted=1932.105537, expected=1881.770020
predicted=1886.250315, expected=1884.089966
predicted=1886.791564, expected=1920.030029
predicted=1918.256115, expected=1923.819946
predicted=1922.899622, expected=1951.359985
predicted=1950.988798, expected=1987.050049
predicted=1982.650174, expected=1979.920044
predicted=1977.183502, expected=1995.829956
predicted=1994.409041, expected=2013.430054
predicted=2009.958186, expected=2014.890015
predicted=2012.885214, expected=2017.459961
predicted=2017.266449, expected=2003.689941
predicted=2003.680673, expected=1994.239990
predicted=1995.012176, expected=2023.859985
predicted=2022.536789, expected=2033.109985
predicted=2031.472041, expected=2033.660034
predicted=2033.918776, expected=2030.770020
predicted=2030.792121, expected=2018.939941
predicted=2018.764410, expected=2052.510010
predicted=2050.784276, expected=2075.149902
predicted=2072.319443, expected=2071.179932
predicted=2070.694241, expected=2065.889893
predicted=2066.220299, expected=2090.350098
predicted=2087.640973, expected=2089.409912
predicted=2087.764002, expected=2079.360107
predicted=2080.395053, expected=2104.050049
predicted=2102.867681, expected=2109.790039
predicted=2107.583081, expected=2102.310059
predicted=2102.854258, expected=2099.929932
predicted=2100.509728, expected=2099.199951
predicted=2098.616976, expected=2078.580078
predicted=2080.095576, expected=2081.719971
predicted=2083.075731, expected=2075.000000
predicted=2075.642984, expected=2045.969971
predicted=2048.786887, expected=2023.040039
predicted=2026.813495, expected=2053.189941
predicted=2052.723757, expected=2050.439941
predicted=2050.142723, expected=2083.580078
predicted=2083.200910, expected=2081.239990
predicted=2080.168259, expected=2089.169922
predicted=2087.800285, expected=2086.590088
predicted=2086.009531, expected=2089.139893
predicted=2088.106309, expected=2088.870117
predicted=2088.900656, expected=2090.110107
predicted=2089.996695, expected=2080.409912
predicted=2081.216379, expected=2102.629883
predicted=2101.788151, expected=2079.510010
predicted=2080.143012, expected=2049.620117
predicted=2052.952163, expected=2091.689941
predicted=2090.628936, expected=2077.070068
predicted=2075.847086, expected=2063.590088
predicted=2066.789089, expected=2047.619995
predicted=2049.848488, expected=2052.229980
predicted=2051.721830, expected=2012.369995
predicted=2015.799946, expected=2021.939941
predicted=2024.292783, expected=2043.410034
predicted=2042.321224, expected=2073.070068
predicted=2070.795230, expected=2041.890015
predicted=2043.930473, expected=2005.550049
predicted=2008.825748, expected=2021.150024
predicted=2021.031943, expected=2038.969971
predicted=2036.798206, expected=2064.290039
predicted=2063.601357, expected=2060.989990
predicted=2060.956939, expected=2056.500000
predicted=2056.100899, expected=2078.360107
predicted=2076.300902, expected=2063.360107
predicted=2062.543959, expected=2043.939941
predicted=2046.278762, expected=2012.660034
predicted=2015.748845, expected=2016.709961
predicted=2017.908600, expected=1990.260010
predicted=1993.007705, expected=1943.089966
predicted=1948.797434, expected=1922.030029
predicted=1927.039327, expected=1923.670044
predicted=1925.244071, expected=1938.680054
predicted=1939.606685, expected=1890.280029
predicted=1894.504640, expected=1921.839966
predicted=1923.126326, expected=1880.329956
predicted=1881.534471, expected=1881.329956
predicted=1883.484553, expected=1859.329956
predicted=1862.289371, expected=1868.989990
predicted=1868.970298, expected=1906.900024
predicted=1905.601371, expected=1877.079956
predicted=1877.551104, expected=1903.630005
predicted=1903.861024, expected=1882.949951
predicted=1882.486838, expected=1893.359985
predicted=1892.876083, expected=1940.239990
predicted=1937.297058, expected=1939.380005
predicted=1936.821430, expected=1903.030029
predicted=1905.925898, expected=1912.530029
predicted=1912.836659, expected=1915.449951
predicted=1913.640718, expected=1880.050049
predicted=1882.965394, expected=1853.439941
predicted=1858.148931, expected=1852.209961
predicted=1853.513685, expected=1851.859985
predicted=1852.544613, expected=1829.079956
predicted=1832.200342, expected=1864.780029
predicted=1864.579719, expected=1895.579956
predicted=1891.989661, expected=1926.819946
predicted=1923.658229, expected=1917.829956
predicted=1917.191039, expected=1917.780029
predicted=1916.619588, expected=1945.500000
predicted=1942.467681, expected=1921.270020
predicted=1920.679561, expected=1929.800049
predicted=1930.839709, expected=1951.699951
predicted=1949.663313, expected=1948.050049
predicted=1946.729236, expected=1932.229980
predicted=1934.276908, expected=1978.349976
predicted=1975.691561, expected=1986.449951
predicted=1983.035072, expected=1993.400024
predicted=1992.999197, expected=1999.989990
predicted=1999.341057, expected=2001.760010
predicted=1999.767201, expected=1979.260010
predicted=1980.419839, expected=1989.260010
predicted=1989.568737, expected=1989.569946
predicted=1988.969496, expected=2022.189941
predicted=2020.426276, expected=2019.640015
predicted=2019.029254, expected=2015.930054
predicted=2016.070518, expected=2027.219971
predicted=2026.417951, expected=2040.589966
predicted=2038.209708, expected=2049.580078
predicted=2048.633994, expected=2051.600098
predicted=2051.182829, expected=2049.800049
predicted=2049.431090, expected=2036.709961
predicted=2037.248000, expected=2035.939941
predicted=2036.484926, expected=2037.050049
predicted=2037.163902, expected=2055.010010
predicted=2054.200047, expected=2063.949951
predicted=2063.129744, expected=2059.739990
predicted=2059.741070, expected=2072.780029
predicted=2072.055074, expected=2066.129883
predicted=2065.358636, expected=2045.170044
predicted=2046.771047, expected=2066.659912
predicted=2066.482656, expected=2041.910034
predicted=2042.298380, expected=2047.599976
predicted=2049.136963, expected=2041.989990
predicted=2042.739886, expected=2061.719971
predicted=2060.422062, expected=2082.419922
predicted=2081.074888, expected=2082.780029
predicted=2081.771738, expected=2080.729980
predicted=2080.943037, expected=2094.340088
predicted=2092.773675, expected=2100.800049
predicted=2099.170621, expected=2102.399902
predicted=2102.173345, expected=2091.479980
predicted=2092.201574, expected=2091.580078
predicted=2091.748114, expected=2087.790039
predicted=2087.995034, expected=2091.699951
predicted=2091.912345, expected=2095.149902
predicted=2095.313163, expected=2075.810059
predicted=2077.143009, expected=2065.300049
predicted=2067.214471, expected=2081.429932
predicted=2080.930911, expected=2063.370117
predicted=2064.085584, expected=2051.120117
predicted=2053.809730, expected=2050.629883
predicted=2051.633510, expected=2057.139893
predicted=2056.595724, expected=2058.689941
predicted=2059.234992, expected=2084.389893
predicted=2083.253794, expected=2064.459961
predicted=2064.615775, expected=2064.110107
predicted=2065.037953, expected=2046.609985
predicted=2047.609662, expected=2066.659912
predicted=2065.724006, expected=2047.209961
predicted=2048.458981, expected=2047.630005
predicted=2048.928544, expected=2040.040039
predicted=2041.134852, expected=2052.320068
predicted=2051.560103, expected=2048.040039
predicted=2048.621922, expected=2076.060059
predicted=2074.620146, expected=2090.540039
predicted=2088.546764, expected=2090.100098
predicted=2089.153770, expected=2099.060059
predicted=2098.477034, expected=2096.949951
predicted=2095.658261, expected=2099.330078
predicted=2098.930537, expected=2105.260010
predicted=2104.862591, expected=2099.129883
predicted=2099.142731, expected=2109.409912
predicted=2109.257308, expected=2112.129883
predicted=2111.505987, expected=2119.120117
predicted=2118.562236, expected=2115.479980
predicted=2115.680904, expected=2096.070068
predicted=2097.317487, expected=2079.060059
predicted=2081.143449, expected=2075.320068
predicted=2076.411806, expected=2071.500000
predicted=2072.523447, expected=2077.989990
predicted=2078.857500, expected=2071.219971
predicted=2072.212469, expected=2083.250000
predicted=2083.139176, expected=2088.899902
predicted=2088.214349, expected=2085.449951
predicted=2085.418391, expected=2113.320068
predicted=2111.925864, expected=2037.410034
predicted=2040.951168, expected=2000.540039
predicted=2006.616174, expected=2036.089966
predicted=2035.652970, expected=2070.770020
predicted=2066.941484, expected=2098.860107
predicted=2098.528575, expected=2102.949951
predicted=2102.017521, expected=2088.550049
predicted=2087.469788, expected=2099.729980
predicted=2098.279919, expected=2097.899902
predicted=2096.666803, expected=2129.899902
predicted=2128.153475, expected=2137.159912
predicted=2135.575505, expected=2152.139893
predicted=2150.481296, expected=2152.429932
predicted=2151.386055, expected=2163.750000
predicted=2161.938842, expected=2161.739990
predicted=2161.033139, expected=2166.889893
predicted=2166.298327, expected=2163.780029
predicted=2163.764888, expected=2173.020020
predicted=2172.416438, expected=2165.169922
predicted=2165.430274, expected=2175.030029
predicted=2174.827600, expected=2168.479980
predicted=2168.634346, expected=2169.179932
predicted=2169.422641, expected=2166.580078
predicted=2167.089251, expected=2170.060059
predicted=2169.936467, expected=2173.600098
predicted=2173.643220, expected=2170.840088
Test RMSE: 16.427

Now we have a much better model (RMSE: 16.427) and no communication challenge. Lets improve the user experience with a better graph:

In [157]:
plotly_actual_vs_predicted(index=ts2.index[last_points_for_testing:].sort_values(), actuals=test, predictions=predictions, title='Actuals vs Predictions')

And now lets add the 95% confidence intervals (I did not add in the first dataset because it is too linear)

In [197]:
common_kw = dict(x=ts2.index[last_points_for_testing:].sort_values(), mode='lines+markers')
xaxis = dict(title='Predictions with confidence interval')

data_actuals = go.Scatter(y=test, name='actuals',  **common_kw)
data_predictions = go.Scatter(y=predictions, name='predictions',  **common_kw)

upper_bound = go.Scatter(y=confidence_upper, name='upper', mode='lines', x=ts2.index[last_points_for_testing:].sort_values(), marker=dict(color="#444"), line=dict(width=0), fillcolor='rgba(68, 68, 68, 0.3)', fill='tonexty')
lower_bound = go.Scatter(y=confidence_lower, name='lower', mode='lines', x=ts2.index[last_points_for_testing:].sort_values(), marker=dict(color="#444"), line=dict(width=0))

data = [lower_bound, upper_bound, data_actuals, data_predictions]
layout = dict(title='Predictions with confidence interval', showlegend=True, xaxis=xaxis)
fig = dict(data=data, layout=layout)

iplot(fig, show_link=False)

Analysing and forecasting the third dataset

Loading the data

In [122]:
dataset3 = pd.read_csv('data3.csv')
dataset3['bond_id'] = dataset3['bond_id'].astype(str)

# There are some missing values into curve_based_price_last1. Because this feature is a TS, I am going to
# use the prior observation to fill missing values
df1 = dataset3.curve_based_price_last1.to_frame()
df1['id'] = df1.index
df2 = dataset3.curve_based_price_last1.shift(1).to_frame()
df2['id'] = df2.index
df_merge = pd.merge(df1, df2, on=['id'], how='inner')
df_merge['curve_based_price_last1'] = np.where(df_merge['curve_based_price_last1_x'].isnull(), df_merge['curve_based_price_last1_y'], df_merge['curve_based_price_last1_x'])
dataset3['curve_based_price_last1'] = df_merge['curve_based_price_last1']

Just curious to see the data. Lets take a look in the end of the serie

In [3]:
xaxis = dict(title='Predictions with confidence interval')

data_actuals = go.Scatter(y=dataset3.tail(500)['trade_price'], name='trade price',  mode='lines+markers')
data = [data_actuals]
layout = dict(title='Predictions with confidence interval', showlegend=True, xaxis=xaxis)
fig = dict(data=data, layout=layout)

iplot(fig, show_link=False)

It looks we have some outliers. We will come back to this topic later, if needed. Lets decompose de serie

In [147]:
# Add date to the dataframe
dataset3['col_dt'] = datetime.datetime.now()
dataset3['linha'] = dataset3.index.astype(int)
dataset3['col_dt'] = dataset3.apply(lambda x: x.col_dt - timedelta(hours = x.linha), axis=1)

# Store my TS on "ts1" object
ts3 = pd.Series(index=dataset3.col_dt, data=dataset3.trade_price.values)

# Check some records
print(ts3.head())

# Perform naive decomposition (check package documentation for more info https://www.statsmodels.org/stable/generated/statsmodels.tsa.seasonal.seasonal_decompose.html
# We can refer to each result by .trend, .seasonal, .resid and .observed
result = seasonal_decompose(ts3, model='additive')
result.plot()
pyplot.show()

Now, we will try to understand the most relevant variables.

Feature importance

While I was doing some research about this topic, I found this interesting package https://tsfresh.readthedocs.io/en/latest/. I will take a look on it in another opportunity, just thought you might be interesting too. Here, I will try to understand the feature importance using my regression coefficients.

We have one target variable "trade_price" and a bunch of independent variables. We got some instructions to give special attention to the "bond_id" variable.

For this example, I am using SelectKBest from sk-learn. This function will show feature importance based on the F-value between label/feature for regression tasks.

In [4]:
features = [                              
'bond_id',                                                
'weight',                           
'current_coupon',                   
'time_to_maturity',                 
'is_callable',                      
'reporting_delay',                  
'trade_size',                       
'trade_type',                       
'curve_based_price',                
'received_time_diff_last1',         
'trade_price_last1',                
'trade_size_last1',                 
'trade_type_last1',                 
'curve_based_price_last1'  
]

df_features = dataset3[features]

# Checking some variable importance
selector = SelectKBest(score_func=f_regression, k='all')
fit = selector.fit(df_features, dataset3['trade_price'])

# Show results
varImp = dict()
for variable, score, topX in zip(df_features.columns, fit.scores_, selector.get_support()):
    varImp[variable] = score
pd.DataFrame.from_dict(varImp, orient='index').to_csv('varImp.csv')

varImp
Out[4]:
{'bond_id': 32224.297061954785,
 'weight': 5236.820846448235,
 'current_coupon': 1772.340783584678,
 'time_to_maturity': 12233.817310886476,
 'is_callable': 48718.28323160084,
 'reporting_delay': 0.120827631732898,
 'trade_size': 2549.6266118936987,
 'trade_type': 3432.4365066054224,
 'curve_based_price': 40371216.96324012,
 'received_time_diff_last1': 938.1062437086148,
 'trade_price_last1': 50232537.61111993,
 'trade_size_last1': 2815.536997751561,
 'trade_type_last1': 3691.4857933288977,
 'curve_based_price_last1': 39291907.93741489}

I have done some manual work to highlight feature importance based on these results. All of these features play a certain level of importance, I built this heat map to show the most important ones (from top to bottom) imp.png

I dont have any business background here, but based on some background that I got from the OLX team (the test itself), it makes a lot of sense. Let's see why:

  • trade_price_last1 is probably the lag -1 of the TS (although -1 might be many things), then it makes sense to be the most important feature
  • I got some insights to use curve_based_price and it is indeed there as important feature (as well as curve_based_price_last1, which is probably similar to the previous bullet explanation)
  • Then I got two categorical variables (binary and multi class, respectively). And again, for one of them, I was oriented to give special attention.

In fact, it might be worth to explore all of them, but I will stop my arguments here due to my limited time. There is a question around the "bond_id" variable. I will try to analyse it's levels.

Analysing "bond_id"

First, we will create a dataset of bonds with enough data (at least more than one record). This is needed just because I will work with a stratified sample of the data. BTW, ideally, I should not be using train and test splits to fo feature engineering, but I will go ahead with this approach.

In [5]:
aggregations = dict()
aggregations['id'] = 'count'
df_bond = dataset3.groupby(['bond_id'], as_index=False).agg(aggregations)
df_bond = df_bond.rename(columns={'id': 'count'})
df_bond = df_bond[df_bond['count'] > 1]
df_bond.head()
Out[5]:
bond_id count
0 1 28
1 10044 32
2 10054 114
3 10077 49
4 10095 13

Now, I will limit my dataset3 by those bonds

In [6]:
# Filter dataset3
dataset3_with_enough_bond = pd.merge(dataset3, df_bond, on=['bond_id'], how='inner')

# Specify the feature space
features = [                              
'bond_id',                                                
'weight',                           
'current_coupon',                   
'time_to_maturity',                 
'is_callable',                      
'reporting_delay',                  
'trade_size',                       
'trade_type',                       
'curve_based_price',                
'received_time_diff_last1',         
'trade_price_last1',                
'trade_size_last1',                 
'trade_type_last1',                 
'curve_based_price_last1'  
]

df_features = dataset3_with_enough_bond[features]

In the next steps I am going to perform a one hot encoder operation in the dataset, using bond_ids as features. That will give me a very sparse matrix. To avoid running out of memory, I will play with a stratified sample of the data.

In [7]:
X_train, X_test, y_train, y_test = train_test_split(df_features, dataset3_with_enough_bond['trade_price'], test_size = 0.5, stratify=df_features.bond_id)
df_features = X_train

Finally, I will go ahead an create my one hot encoder variables using my sample dataset

In [ ]:
# Create a key column just to facilitate the final merge in the end of this section
df_features['id'] = df_features.index

###
### PART 1 - CREATE A FACTOR VARIABLE (LABEL ENCODER)
###

le_classificacao = preprocessing.LabelEncoder()
le_classificacao.fit(df_features.bond_id)
df_features['le_classificacao'] = le_classificacao.transform(df_features.bond_id)
dump(le_classificacao, open('le_classificacao.sav', 'wb'))

###
### PART 2 - DUMMIES
###

# Create a new dataset with key + input variable
dataset_new = df_features[['id', 'le_classificacao']]

# OneHotEncoder
encoder = OneHotEncoder(categorical_features = np.array([False, True]), dtype=bool, sparse=True)
encoder.fit(dataset_new)

# Save for future use
dump(encoder, open('encoder.sav', 'wb'))

# Prepare column names
colnames = list(le_classificacao.classes_)
colnames.append('id')

# Execute the transformation in the dataset
results = encoder.transform(dataset_new)
dataset_encode = pd.DataFrame(results.toarray(), columns=colnames)

# Convert data types
dataset_encode = dataset_encode.astype(str)
dataset_encode['id'] = dataset_encode['id'].astype(float)

# Merge final results
dataset_merge = pd.merge(df_features, dataset_encode, on='id', how='inner')

# Check results
dataset_merge.head()

Doing some clean up before testing variable importance

In [ ]:
df_one_hot_encoded = dataset_merge.drop(['id', 'bond_id', 'le_classificacao'] ,axis=1)
df_one_hot_encoded.dtypes

Checking some variable importance again, this time considering my one hot encoder variables

In [ ]:
selector = SelectKBest(score_func=f_regression, k='all')
fit = selector.fit(df_one_hot_encoded, y_train)

# Show results
varImp = dict()
for variable, score, topX in zip(df_one_hot_encoded.columns, fit.scores_, selector.get_support()):
    varImp[variable] = score
pd.DataFrame.from_dict(varImp, orient='index').to_csv('varImp.csv')

varImp

imp_bond.png

It looks there are a few bonds more important than others.

Building the final model

I will prepare my modeling dataset. Instead of using sklearn encoder, I decided to create my dummy variables manually.

In [90]:
modeling_df = dataset3[['trade_price', 'bond_id', 'trade_price_last1', 'curve_based_price', 'curve_based_price_last1', 'is_callable']]
modeling_df['is_348003'] = np.where(modeling_df['bond_id'] == '348003', '1', '0')
modeling_df['is_195002'] = np.where(modeling_df['bond_id'] == '195002', '1', '0')
modeling_df['is_callable'] = modeling_df['is_callable'].astype(str)
modeling_df = modeling_df.drop(['bond_id'], axis=1)
modeling_df.head()
Out[90]:
trade_price trade_price_last1 curve_based_price curve_based_price_last1 is_callable is_348003 is_195002
0 128.596 129.089 127.561201 129.435587 0 0 0
1 126.673 128.596 126.323528 127.561201 0 0 0
2 126.880 126.673 126.761433 126.323528 0 0 0
3 126.362 126.880 127.025765 126.761433 0 0 0
4 124.918 126.362 125.736081 127.025765 0 0 0
In [91]:
# Separate target and features
target = modeling_df['trade_price']
datasetFiltered = modeling_df.drop('trade_price', 1)
In [75]:
X = datasetFiltered.values
Y = target

# split the data into train and test (in this case I am selecting the last 25% of the data for testing)
number_points = len(X)
last_points_for_testing = int(number_points * 0.25)

print ('-------Using {} rows for testing'.format(last_points_for_testing))

X_train, X_test = X[1:len(X)-last_points_for_testing], X[len(X)-last_points_for_testing:]
Y_train, Y_test = Y[1:len(Y)-last_points_for_testing], Y[len(Y)-last_points_for_testing:]
-------Using 190669 rows for testing
In [92]:
# Create the model
baseline = LinearRegression()
baseline.fit(X_train, Y_train)
Out[92]:
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
In [93]:
# r2
print(baseline.score(X_train, Y_train))

# r2 adjusted
print (1 - (1-baseline.score(X_train, Y_train))*(len(Y_train)-1)/(len(Y_train)-X_train.shape[1]-1))
0.9894507331550288
0.9894506224985771
In [78]:
# Testa contra os dados de teste
predictions = baseline.predict(X_test)
In [79]:
# MSE
mean_squared_error(Y_test, predictions)
Out[79]:
0.9835215217997957

So far my metrics looks ok. I will finish looking to the residuals

In [87]:
pyplot.plot(Y_test - predictions)
Out[87]:
[<matplotlib.lines.Line2D at 0x7f40e21a7128>]
In [88]:
pd.DataFrame(Y_test - predictions).plot(kind='density')
Out[88]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f40e2585048>

They look random and normal distributed. Good point then.

Lets finally visualize results.

In [84]:
plotly_actual_vs_predicted(actuals=Y_test[len(Y_test)-500:len(Y_test),], predictions=predictions[len(Y_test)-500:len(Y_test),], title='Actuals vs Predictions')

Since our predictions looks good, I will pack this model and make it available on my Flask API.

Thanks

In [109]:
filename = 'model.sav' 
dump(baseline, open(filename, 'wb'))